π Getting Started with Ferry β
Welcome to Ferry β΄οΈ, the lightweight and powerful data ingestion tool. Whether you're pulling data from a database, an API, or a file, Ferry makes ingestion simple, fast, and observable.
This guide will walk you through installation and your first data ingestion example.
π οΈ Installation β
Ferry can be installed using pip:
sh
pip install ferry
Verify the installation:
sh
ferry --version
β‘ Your First Data Ingestion β
Let's use cURL with Ferryβs HTTP API to ingest data. In this example, we'll transfer data from a PostgreSQL
database to a ClickHouse
data warehouse.
Step 1: Start the Ferry Server β
Run the Ferry service locally:
sh
ferry serve
Step 2: Send Data Using cURL β
Use cURL to transfer data from a PostgreSQL
database to a ClickHouse
data warehouse.
sh
curl -X POST http://localhost:8000/ingest \
-H "Content-Type: application/json" \
-d '{
"identity": "fgXOw4zY"
"source_uri": "postgresql://postgres:password@localhost:5432/db_name",
"destination_uri": "clickhouse://default:password@localhost:9000/db_name?http_port=8123&secure=0",
"resources": [
{"source_table_name": "users"}
]
}'
Required parameters:
identity
: a unique identifier for the ingestionsource_uri
: the source database uridestination_uri
: the destination database uriresources
: a list of resources you want to ingest