Meet Ingestr: A Command-Line Application that Allows You to Ingest Data from Any Source into any Destination with No Code Necessary

In the world of managing data, getting data from different places to where you want it can be tricky. Ingesting data from various sources into different destinations can be a complex and time-consuming process, often requiring custom code and backend management. The process, essential for data-driven decision-making and analysis, often involves complex backend configurations or custom code, presenting significant challenges for users.

Traditionally, individuals tasked with data ingestion have had to grapple with intricate backend setups or write custom code, leading to time-consuming and error-prone processes. Furthermore, managing incremental loading, such as appending, merging, or deleting and inserting data, adds another layer of complexity to the task.

Meet Ingestr: a command-line application poised to revolutionize the data ingestion landscape. With its intuitive interface and simple command-line flags, Ingestr eliminates the need for intricate backend management or coding expertise. Users can effortlessly copy data from databases or other sources into their desired destinations, all with a single command.

One of Ingestr's standout features is its support for incremental loading, allowing users to seamlessly append, merge, or perform delete-insert operations. This capability streamlines data updating processes, ensuring that users can keep their datasets up-to-date with minimal effort. Ingestr's lightweight nature and Python-friendly syntax make it highly adaptable and accessible across different platforms and tasks. Its efficient optimization algorithms enable swift data ingestion, while its quick visualization capabilities provide users with insights into the ingestion process, enhancing decision-making.

In conclusion, Ingestr emerges as a game-changer in the realm of data ingestion, simplifying a once complex and time-consuming task into a seamless and efficient process. By automating data movement and management, Ingestr empowers users to focus on deriving insights and value from their data, ushering in a new era of data-driven decision-making. As organizations continue to harness the power of data, tools like Ingestr will undoubtedly play a pivotal role in unlocking its full potential.

Installation

pip install ingestr

Quickstart

ingestr ingest \
    --source-uri 'postgresql://admin:admin@localhost:8837/web?sslmode=disable' \
    --source-table 'public.some_data' \
    --dest-uri 'bigquery://<your-project-name>?credentials_path=/path/to/service/account.json' \
    --dest-table 'ingestr.some_data'