Tool for migrating data from MSSQL to AWS Redshift part 3 / 3

This tool is now mostly finished. I just need to finish the integration tests for Pytest testing module. Otherwise please feel free to clone, fork or contribute.

You can find it here:

https://github.com/datahappy1/mssql_to_redshift_data_transfer_tool

 

Advertisements

Dummy .csv or flat .txt file generator in Python 3.7

I just finished my dummy csv or flat text file generator written in Python 3.7.

In my opinion, such project is quite unique. I use this tool for large files generation, so I can do performance testing loads in ETL data-ingestion pipelines without loading production data in Dev / Test environments or without the need to de-identify PII.

Feel free to clone, fork or contribute with new features and feedback.

The project is located here:

https://github.com/datahappy1/dummy_file_generator

 

In the future,I’d like to make it also AWS Server-less design event driven project, so stay tuned 🙂