etlunit Package

etlunit Package

The etlUnit package!

This package is the main package of the project. It houses all of the submodules necessary for building the test application.

code_executor Module

This file houses all of the code necessary to execute the application that we are generating.

class etlunit.code_executor.CodeExecutor(out_dir)

Class that executes the code that we built from the templates.

execute(test)

This method actually performs the execution of the code we generated. :param test: A boolean that determins if this is a test execution or not. If it is a test, then we do not really execute the code, we only print a list of files that would have been executed. :type test: bool.

code_generator Module

This file houses all of the code necessary to generate code from templates.

class etlunit.code_generator.CodeGenerator(out_dir, data)

This class performs the generation of the code. Using the Jinja2 template engine, we are taking in YAML and generating code from it by filling in templates.

generateCode(test)

This method actually generates the code. :param test: A boolean that determines if we are running a test or not. If its true, then we don’t persist the code that we generate, it prints to stdout instead. :type test: bool.

persist_output(name, output, test)

This method persist the generated code to the output directory specified. :param name: The name of the test suite. :type name: str. :param output: The output from the template being rendered. :type output: str. :param test: A boolean that determines if we are testing or not. If we are testing, then output is not persisted. :type test: bool.

etlUnit Module

This is the main file for the application.

etlunit.etlUnit.main(argv)

This class is the entry point for the application. It takes the arguments, validates them, and passes them on to the appropriate classes to continue execution.

There are three main functions of this application. 1) Take in YAML 2) Generate code from that YAML 3) Execute that code so that we can take advantage of the unittest libraries

yaml_reader Module

This file houses all of the code to read in the YAML files.

class etlunit.yaml_reader.YAMLReader(in_file, in_dir)

Class to read YAML files.

readFile(filename)

This method actually reads the files contents and adds them an array for later consumption.

Parameters:filename (str.) – The file name that we are reading.

readTests()

This method determines if were reading a file or a directory, then calls the readFile method.

Subpackages

• template_base Package
  • template_base Package
  • connections_reader Module
  • sqlalchemy_connector Module
• test Package
  • test Package
  • test_etlunit Module
• utils Package
  • utils Package
  • custom_logging Module
  • settings Module