Local dev

Set up Velvet with your local development environment

Add your local development database as a remote connection in Velvet, and query against it.

Connect your local database using ngrok

ngrok is a unified ingress platform for developers to test and preview applications and APIs. Follow this guide to set up ngrok with your local development environment.

  1. Install ngrok here.
  2. Authenticate ngrok in your terminalsh ngrok authtoken <your_auth_token>.
  3. Identify the port your local development database is running on.
  4. Start an ngrok tunnel to your database's port using the following command sh ngrok tcp <database_port>.
  5. Once the tunnel is up, ngrok will display a forwarding address that looks something like tcp://0.tcp.ngrok.io:12345. This address consists of a randomly generated domain and port that will forward traffic to your local database port through the tunnel.
  6. To connect your database to Velvet, use the forwarding address and port provided by ngrok as the host and port in your database connection string. You'll also need to provide the appropriate username, password, and database name as required by your database.

See further instructions and connection string syntax below for each database.

PostgreSQL

Create the connection string

Given the example forwarding address: tcp://[0.tcp.ngrok.io](http://0.tcp.ngrok.io):12345 the host will be 0.tcp.ngrok.io and the port is 12345.

Assuming the database name your_db_name, the username your_username and the password is your_password, your connection string should be postgresql://your_username:[email protected]:12345/your_db_name.

Supabase

To connect a local Supabase DB, you'll need to run a local version of Supabase.

Run Supabase Locally

  1. Ensure you have Docker and Docker Compose installed on your machine. Docker is used to containerize and run the services Supabase requires, such as PostgreSQL, Auth, and Storage.

  2. Supabase maintains a GitHub repository with a Docker Compose file that you can use to start Supabase locally. You can clone this repository or just download the relevant docker-compose.yml file. The repo you're looking for is supabase/supabase. They provide a quickstart command for cloning with just the necessary files:

    git clone --depth 1 https://github.com/supabase/supabase
    cd supabase
    
  3. Once inside the cloned directory, use Docker Compose to start all the required services. The following command starts all the necessary services defined in the docker-compose.yml, including the Postgres database, Auth service, Realtime for subscription services, and Storage:

    docker-compose up
    
  4. After starting the services, you can access the Supabase Studio (the web interface for managing your Supabase project) through your browser, usually at http://localhost:3000 or another port specified in the Docker Compose output.

Create the connection string

Given the example forwarding address: tcp://[0.tcp.ngrok.io](http://0.tcp.ngrok.io):12345 the host will be 0.tcp.ngrok.io and the port is 12345.

Assuming the database name your_db_name, the username your_username and the password is your_password, your connection string should be postgresql://your_username:[email protected]:12345/your_db_name.

Fly

Fly.io's PostgreSQL offering, often referred to as "Fly Postgres," is designed to run on Fly.io's infrastructure, leveraging their global network for high availability and performance.

To mimic or work with a PostgreSQL database setup locally that resembles what you'd deploy on Fly.io, instal PostgreSQL on your local machine and configure it similarly to your Fly.io setup.

Run PostgreSQL Locally:

  1. Download and install PostgreSQL from the official website or use a package manager for your operating system ( apt on Ubuntu, brew on macOS, or choco on Windows).
  2. Set up your local PostgreSQL instance to match the configuration you use on Fly.io as closely as possible. This includes creating databases, users, and configuring any necessary extensions.

Create the connection string

Given the example forwarding address: tcp://[0.tcp.ngrok.io](http://0.tcp.ngrok.io):12345 the host will be 0.tcp.ngrok.io and the port is 12345.

Assuming the database name your_db_name, the username your_username and the password is your_password, your connection string should be postgresql://your_username:[email protected]:12345/your_db_name.

MongoDB

Create the connection string

Given the example forwarding address: tcp://[0.tcp.ngrok.io](http://0.tcp.ngrok.io):12345 the host will be 0.tcp.ngrok.io and the port is 12345.

Assuming the database name your_db_name, the username your_username and the password is your_password, your connection string should be mongodb://your_username:[email protected]:12345/your_db_name.

MySQL

Create the connection string

Given the example forwarding address: tcp://[0.tcp.ngrok.io](http://0.tcp.ngrok.io):12345 the host will be 0.tcp.ngrok.io and the port is 12345.

Assuming the database name your_db_name, the username your_username and the password is your_password, your connection string should be myql://your_username:[email protected]:12345/your_db_name.

Snowflake

To connect a local Snowflake instance you'll need to run a local version of Snowflake.

There is no direct Snowflake container or local version. Developers sometimes use Docker containers running databases like Postgres or MySQL to simulate Snowflake’s SQL capabilities. This approach won't replicate Snowflakes architecture or performance, but can be used for SQL testing.

Run PostgreSQL or MySQL Locally:

  1. Ensure you have Docker and Docker Compose installed on your machine. Run a PostgresQL or MySQL container:

For PostgreSQL:

docker run --name my-database -e POSTGRES_PASSWORD=mypassword -d postgres
  • The command above creates a PostgreSQL database named my-database with password mypassword .
  • PostgreSQL uses the default port 5432.

For MySQL:

docker run --name my-database -e MYSQL_ROOT_PASSWORD=mypassword -d mysql:tag
  • The command above creates a MySQL database named my-database with password mypassword .
  • MySQL uses the default port 3306.

Create the connection string

Given the example forwarding address: tcp://[0.tcp.ngrok.io](http://0.tcp.ngrok.io):12345 the host will be 0.tcp.ngrok.io and the port is 12345.

Assuming the database name your_db_name, the username your_username and the password is your_password, your connection string should be postgresql://your_username:[email protected]:12345/your_db_name.

S3

To connect a local S3 instance, you'll need to run a local version of S3.

Run S3 locally using MinIO

  1. Ensure you have Docker and Docker Compose installed on your machine.

  2. MinIO is an open-source tool designed to provide an AWS S3 compatible object storage system.

  3. Open a terminal and run the following command to pull the latest MinIO Docker image:

    docker pull minio/minio
    
  4. Start a MiniO instance as an S3 server using the following command:

    mkdir -p ~/minio/data
    
    docker run \
       -p 9000:9000 \
       -p 9001:9001 \
       --name minio \
       -v ~/minio/data:/data \
       -e "MINIO_ROOT_USER=ROOTNAME" \
       -e "MINIO_ROOT_PASSWORD=CHANGEME123" \
       quay.io/minio/minio server /data --console-address ":9001"
    

    mkdir creates a new local directory at ~/minio/data in your home directory.

    docker run starts the MinIO container.

    p binds a local port to a container port.

    name creates a name for the container.

    v sets a file path as a persistent volume location for the container to use. When MinIO writes data to /data, that data mirrors to the local path ~/minio/data, allowing it to persist between container restarts. You can replace ~/minio/data with another local file location to which the user has read, write, and delete access.

    e sets the environment variables [MINIO_ROOT_USER](https://min.io/docs/minio/linux/reference/minio-server/settings/root-credentials.html#envvar.MINIO_ROOT_USER) and [MINIO_ROOT_PASSWORD](https://min.io/docs/minio/linux/reference/minio-server/settings/root-credentials.html#envvar.MINIO_ROOT_PASSWORD). These set the root user credentials. Change the example values to use for your container.

  5. Open a web browser and navigate to http://localhost:9001 to access the MinIO console. Log in with the MINIO_ROOT_USER and MINIO_ROOT_PASSWORD you specified earlier.

  6. Once logged in you can create buckets, upload files and manager your data similarly to how you would on AWS S3.

  7. Use database port 9001 in the next section regarding ngrok.

Create the connection string

Given the example forwarding address: tcp://[0.tcp.ngrok.io](http://0.tcp.ngrok.io):12345 the host will be 0.tcp.ngrok.io and the port is 12345.

Assuming the database name your_db_name, the username your_username and the password is your_password, your connection string should be postgresql://your_username:[email protected]:12345/your_db_name.