Skip to content

Deploy ReadySet with Kubernetes

This page shows you how to run ReadySet with Kubernetes on Amazon EKS in front of an Amazon RDS Postgres or MySQL database.

Tip

If you don't want to run and manage ReadySet yourself, get a fully-managed deployment on ReadySet Cloud.

Before you begin

  • Note that this tutorial covers the scale-out deployment pattern, with the ReadySet Server and Adapter running as separate processes on separate machines.

  • Make sure you have an Amazon RDS for Postgres database running Postgres 13 or 14.

    If you want to integrate with another version of Postgres, please contact ReadySet.

  • Make sure there are no DDL statements in progress.

    ReadySet will take an initial snapshot of your data. Until the entire snapshot is finished, which can take between a few minutes to several hours depending on the size of your dataset, DDL statements (e.g., ALTER and DROP) against tables in your snapshot will be blocked.

  • Make sure tables without primary keys have REPLICA IDENTITY FULL.

    If the database you want ReadySet to replicate includes tables without primary keys, make sure you alter those tables with REPLICA IDENTITY FULL before connecting ReadySet. Otherwise, Postgres will block writes and deletes on those tables.

  • Make sure row-level security is disabled. ReadySet does not currently support row-level security.

  • Complete the steps described in the EKS Getting Started documentation.

    This includes installing and configuring eksctl, the command-line tool for creating and deleting Kubernetes clusters on EKS, and kubectl, the command-line tool for managing Kubernetes from your workstation.

  • Make sure you meet the EKS requirements for using an existing VPC.

    For efficient networking and security, you'll deploy your Kubernetes cluster into the same VPC as your database.

  • Note that this tutorial covers the scale-out deployment pattern, with the ReadySet Server and Adapter running as separate processes on separate machines.

  • Make sure you have an Amazon RDS for MySQL database.

    ReadySet can be run in front of other versions of Postgres and MySQL. However, this tutorial focuses on RDS.

  • Make sure there are no DDL statements in progress.

    ReadySet will take an initial snapshot of your data. Until the entire snapshot is finished, which can take between a few minutes to several hours depending on the size of your dataset, DDL statements (e.g., ALTER and DROP) against tables in your snapshot will be blocked. INSERT and UPDATE statements will also be blocked, but only while a given table is being snapshotted.

  • Complete the steps described in the EKS Getting Started documentation.

    This includes installing and configuring eksctl, the command-line tool for creating and deleting Kubernetes clusters on EKS, and kubectl, the command-line tool for managing Kubernetes from your workstation.

  • Make sure you meet the EKS requirements for using an existing VPC.

    For efficient networking and security, you'll deploy your Kubernetes cluster into the same VPC as your database.

Step 1. Start Kubernetes

In this step, you'll create a Kubernetes cluster on Amazon EKS in the same VPC as your database. Your cluster will contain 3 nodes to accommodate a simple ReadySet deployment of one ReadySet Server, one ReadySet Adapter, and one instance of Consul.

For more demanding workloads, ReadySet can be run with multiple Adapters. Please reach out to ReadySet for guidance.

  1. Identify the subnets in your database's VPC:

    1. In the RDS Console, select your database.
    2. Under Connectivity & security, note the Subnets.

  2. Identify the region where your database is running:

    1. In the RDS Console, select your database.
    2. Under Summary, note the region portion of Region & AZ. For example, us-east-1 is the region portion of us-east-1f.

  3. From your local workstation, create a Kubernetes cluster, replacing the <db-region and <db-subnet> placeholders with the details from the previous steps:

    eksctl create cluster \
--name=readyset \
--region=<db-region> \
--nodegroup-name=standard-workers \
--nodes=3 \
--node-type=c5.2xlarge \
--node-private-networking \
--vpc-private-subnets=<db-subnet1>,<db-subnet2>,<db-subnet3>,...
    Flag Description
    --name The name of the cluster.
    --region The region where your database is running and where you will run your EKS cluster. It is necessary to run your cluster in the same region as your database in order to have access to the same VPC.
    --nodegroup-name

    The name of the node group for the cluster.

    This node group should be created automatically along with the cluster. However, if cluster creation fails because a node group could not be created, you will need to create a managed node group manually and reference it in the --nodegroup-name flag.
    --nodes

    The number of nodes in the cluster.

    3 is the minimum required for a simple ReadySet deployment of one ReadySet Server, one ReadySet Adapter, and one instance of Consul.
    --node-type

    The instance type to use for the nodes.

    The c5.2xlarge type is fine for testing ReadySet; however, ReadySet is a memory-intensive application, so you should use memory-optimized instances (r5.2xlarge or larger) for production deployments.
    --vpc-private-subnets

    The subnets of your database's VPC.

    If you do not want to create the cluster in the same VPC as your database (e.g., you plan to set up VPC peering between Kubernetes and the database), remove this flag and --node-private-networking.

    Cluster provisioning usually takes between 10 and 15 minutes. Do not move on to the next step until you see a message like [✔] EKS cluster "readyset" in "us-east-1" region is ready and details about your cluster.

    Tip

    If cluster creation fails, you may need to create a managed node group manually and reference it in the --nodegroup-name flag.

  4. Check that you can connect to the database from your EKS cluster.

    1. In your EKS cluster, create a temporary pod containing the psql client:

      kubectl run rs-postgres-client \
--rm --tty -i \
--restart='Never' \
--image=postgres \
--namespace=default \
--command -- bash

    2. Start psql, replacing placeholders with your database connection details:

      PGPASSWORD=<password> psql \
--host=<database_endpoint> \
--port=<port> \
--username=<username> \
--dbname=<database_name>

      Tip

      To find the database endpoint, select your database in the RDS Console, and look under Connectivity & security.

      You should now be in the SQL shell, where you can query your database.

      Warning

      If you can't connect, there are likely errors in the psql connection details or in your VPC configuration. Review these details, fix any errors, and try running psql again.

      Do not move on to the next step until you successfully connect from your EKS cluster; if you can't do so now, ReadySet won't be able to connect later.

    3. Stop psql and delete the temporary pod:

      \q

      exit

    1. In your EKS cluster, create a temporary pod containing the mysql client:

      kubectl run rs-mysql-client \
--rm --tty -i \
--restart='Never' \
--image=docker.io/bitnami/mysql:8.0.30-debian-11-r6 \
--namespace=default \
--command -- bash

    2. Start mysql, replacing placeholders with your database connection details:

      mysql \
--host=<database_endpoint> \ 
--port=<port> \
--user=<username> \
--password=<password> \
--database=<database_name>

      Tip

      To find the database endpoint, select your database in the RDS Console, and look under Connectivity & security.

      You should now be in the SQL shell, where you can query your database.

      Warning

      If you can't connect, there are likely errors in the mysql connection details or in your VPC configuration. Review these details, fix any errors, and try running mysql again.

      Do not move on to the next step until you successfully connect from your EKS cluster; if you can't do so now, ReadySet won't be able to connect later.

    3. Stop mysql and delete the temporary pod:

      \q

      exit

  5. Create a Kubernetes secret with your database connection details. ReadySet will use this secret to connect to the database.

    1. Set environment variables with your database connection details:

      Replication scope

      By default, ReadySet will replicate all tables in all schemas of the database specified in DB_NAME. If the queries you want to cache with ReadySet touch only a specific schema or specific tables in a schema, you can restrict the scope of replication accordingly. See Step 4 for more details.

      export DB_USERNAME="<username>"

      export DB_PASSWORD="<password>"

      export DB_NAME="<database_name>"

      export RDS_ENDPOINT="<database_endpoint>"

      Tip

      To find the database endpoint, select your database in the RDS Console, and look under Connectivity & security.

      export CONN_STRING="postgresql://${DB_USERNAME}:${DB_PASSWORD}@${RDS_ENDPOINT}:5432/${DB_NAME}"

    2. Create the secret:

      kubectl create secret \
  generic \
  readyset-db-url \
  --from-literal=url="${CONN_STRING}" \
  --from-literal=username="${DB_USERNAME}" \
  --from-literal=database="${DB_NAME}" \
  --from-literal=password="${DB_PASSWORD}"

      secret/readyset-db-url created

    1. Set environment variables with your database connection details:

      Replication scope

      By default, ReadySet will replicate all tables in the database specified in DB_NAME. If the queries you want to cache with ReadySet touch only specific tables in the database, you can restrict the scope of replication accordingly. See Step 4 for more details.

      export DB_USERNAME="<username>"

      export DB_PASSWORD="<password>"

      export DB_NAME="<database_name>"

      export RDS_ENDPOINT="<database_endpoint>"

      Tip

      To find the database endpoint, select your database in the RDS Console, and look under Connectivity & security.

      export CONN_STRING="mysql://${DB_USERNAME}:${DB_PASSWORD}@${RDS_ENDPOINT}:3306/${DB_NAME}"

    2. Create the secret:

      kubectl create secret \
  generic \
  readyset-db-url \
  --from-literal=url="${CONN_STRING}" \
  --from-literal=username="${DB_USERNAME}" \
  --from-literal=database="${DB_NAME}" \
  --from-literal=password="${DB_PASSWORD}"

      secret/readyset-db-url created

Step 2. Set up load balancing

In this step, you'll install an AWS Network Load Balancer Controller into your Kubernetes cluster. When you deploy ReadySet with the Helm chart, Kubernetes will use this Controller to provision a load balancer for your deployment. The load balancer will be able to handle queries sent to ReadySet from outside of the Kubernetes cluster.

  1. Complete the installation steps described in the AWS Network Load Balancer Controller documentation.

  2. Verify that the network load balancer controller is installed:

    kubectl get deployment -n kube-system aws-load-balancer-controller

    NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
aws-load-balancer-controller   2/2     2            2           84s

Step 3. Configure your database

In this step, you'll configure your database so that ReadySet can consume the database's replication stream, which ReadySet uses to keep its cache up-to-date as the database changes.

  1. In your EKS cluster, create a temporary pod containing the psql client:

    kubectl run rs-postgres-client \
--rm --tty -i \
--restart='Never' \
--image=postgres \
--namespace=default \
--command -- bash

  2. Start psql, replacing placeholders with your database connection details:

    PGPASSWORD=<password> psql \
--host=<database_endpoint> \ 
--port=<port> \
--username=<username> \
--dbname=<database_name>

    Tip

    To find the database endpoint, select your database in the RDS Console, and look under Connectivity & security.

  3. In the psql shell, check if replication is enabled:

    SELECT name,setting
  FROM pg_settings
  WHERE name = 'rds.logical_replication';

    If replication is already on, skip to Step 4. Start ReadySet:

            name             | setting
-------------------------+---------
rds.logical_replication  | on
(1 row)

    If replication is off, continue to the next step:

            name             | setting
-------------------------+---------
rds.logical_replication  | off
(1 row)

  4. Create a custom parameter group.

    • For Parameter group family, select the Postgres version of your database.
    • For Type, select DB Parameter Group.
    • Give the group a name and description.

  5. Edit the new parameter group and set the rds.logical_replication parameter to 1.

  6. Associate the parameter group to your database.

    • Be sure to use the Apply Immediately option. The database must be rebooted in order for the parameter group association to take effect.

    • Do not move on to the next step until the database Status is Available in the RDS Console.

  7. Back in the SQL shell, verify that replication is now enabled:

    SELECT name,setting
  FROM pg_settings
  WHERE name = 'rds.logical_replication';

            name             | setting
-------------------------+---------
rds.logical_replication  | on
(1 row)

    Note

    If replication is still not enabled, reboot the database.

    Once the database Status is Available in the RDS Console, check replication again.

  8. Stop psql and delete the temporary pod:

    \q

    exit

  1. In RDS MySQL, replication is enabled only when automated backups are also enabled. If you didn't enable automated backups when creating your database instance, enable automated backups now.

    • Be sure to use the Apply Immediately option. The database must be rebooted in order for the change to take effect.

    • Do not move on to the next step until the database Status is Available in the RDS Console.

  2. In your EKS cluster, create a temporary pod containing the mysql client:

    kubectl run rs-mysql-client \
--rm --tty -i \
--restart='Never' \
--image=docker.io/bitnami/mysql:8.0.30-debian-11-r6 \
--namespace=default \
--command -- bash

  3. Start mysql, replacing placeholders with your database connection details:

    mysql \
--host=<database_endpoint> \ 
--port=<port> \
--user=<username> \
--password=<password> \
--database=<database_name>

    Tip

    To find the database endpoint, select your database in the RDS Console, and look under Connectivity & security.

  4. In the mysql shell, verify that replication is enabled:

    SHOW VARIABLES LIKE 'log_bin';

    +---------------+-------+
| Variable_name | Value |
+---------------+-------+
| log_bin       | ON    |
+---------------+-------+
1 row in set (0.00 sec)

    Note

    If replication is still not enabled, reboot the database.

    Once the database Status is Available in the RDS Console, check replication again.

  5. Check the binary logging format:

    SHOW VARIABLES LIKE 'binlog_format';

    If the binary logging format is ROW, skip to Step 4. Start ReadySet:

    +---------------+-------+
| Variable_name | Value |
+---------------+-------+
| binlog_format | ROW   |
+---------------+-------+
1 row in set (0.00 sec)

    If the binary logging format is not ROW, continue to the next step:

    +---------------+-------+
| Variable_name | Value |
+---------------+-------+
| binlog_format | MIXED |
+---------------+-------+
1 row in set (0.00 sec)

  6. Create a custom parameter group.

    • For Parameter group family, select the MySQL version of your database.
    • For Type, select DB Parameter Group.
    • Give the group a name and description.

  7. Edit the new parameter group and set the binlog_format parameter to ROW.

  8. Associate the parameter group to your database.

    • Be sure to use the Apply Immediately option. The database must be rebooted in order for the parameter group association to take effect.

    • Do not move on to the next step until the database Status is Available in the RDS Console.

  9. Back in the SQL shell, verify that the binary logging format is ROW:

    SHOW VARIABLES LIKE 'binlog_format';

    +---------------+-------+
| Variable_name | Value |
+---------------+-------+
| binlog_format | ROW   |
+---------------+-------+
1 row in set (0.00 sec)

    Note

    If the binary logging format is still not ROW, reboot the database.

    Once the database Status is Available in the RDS Console, check the binary logging format again.

  10. Stop msql and delete the temporary pod:

    \q

    exit

Step 4. Configure ReadySet

In this step, you'll download and edit the configuration files for deploying ReadySet.

  1. Clone the readyset GitHub repository:

    git clone git@github.com:readysettech/readyset.git

  2. Move into to the readysettech/readyset/helm/readyset directory. This directory contains the Chart.yaml and values.yaml files that Helm needs to deploy ReadySet.

  3. Edit the values.yaml file as follows:

    1. Choose a unique identifier for your ReadySet deployment:

      config:

  # -- Name of the ReadySet deployment. Should be unique within the context
  # of the chosen Consul cluster to avoid key collisions.
  deploymentName: "readyset-helm-test""

    2. Change the image tags for the ReadySet Server and Adapter from latest to the latest release of the ReadySet Server and Adapter (e.g., beta-2023-01-18):

      Note

      The latest docker image tag is updated nightly and so represents different versions of ReadySet over time. To ensure that you deploy a fixed version of ReadySet, it's important to use the tag for a specific version. It's also important the same tag for the ReadySet Server and Adapter.

      # -- Container image settings for ReadySet server.
# @default -- Truncated due to length.
image:

  # -- Container image repository to use for ReadySet server.
  repository: public.ecr.aws/readyset/readyset-server

  # -- Container image tag to use for ReadySet server.
  tag: "latest"

      # -- Container image settings for ReadySet adapter containers.
# @default -- Truncated due to length.
image:

  # -- Image repository to use for ReadySet adapter containers.
  repository: public.ecr.aws/readyset/readyset

  # -- Image tag to use for ReadySet adapter containers.
  tag: "latest"

    3. Configure your deployment to use the ReadySet Adapter for your database:

      # -- Flag to instruct readyset binary which adapter binary to use.
# -- Also used to configure listening port for the helm chart.
# Supported values: mysql, postgresql
database_type: "postgresql"

      # -- Entrypoint arguments for ReadySet adapter containers.
# -- database-type possible values: mysql, postgresql
args: ["--prometheus-metrics", "--database-type", "postgresql"]

      # -- Flag to instruct readyset binary which adapter binary to use.
# -- Also used to configure listening port for the helm chart.
# Supported values: mysql, postgresql
database_type: "mysql"

      # -- Entrypoint arguments for ReadySet adapter containers.
# -- database-type possible values: mysql, postgresql
args: ["--prometheus-metrics", "--database-type", "mysql"]

    4. Change the storage size to be 2x the size of your database:

      volumeClaimTemplates:
- metadata:
    name: state
  spec:
    storageClassName: gp2
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 250Gi

      Note

      The values.yaml file contains the CPU, memory, and storage specifications for the components of your deployment. The default values are suitable for testing purposes only. For production deployments, you'll need to substitute values that are appropriate for your database and workload. Please reach out to ReadySet for guidance.

    5. Set environment variables to disable verification of SSL certifications on the ReadySet Server and ReadySet adapter. This is necessary because ReadySet cannot currently verify Amazon's self-signed certificates.

      # -- Static environment variables to be applied to ReadySet server containers.
extraEnvironmentVars:

  DISABLE_UPSTREAM_SSL_VERIFICATION: "1"

      # -- Static environment variables applied to ReadySet adapter containers.
extraEnvironmentVars:

  DISABLE_UPSTREAM_SSL_VERIFICATION: "1"

    6. By default, ReadySet will replicate all data in the database specified in the ReadySet secret that you created earlier. However, if the queries you want to cache with ReadySet access only a subset of tables in the database, you can set the REPLICATION_TABLES environment variable to narrow the scope accordingly. Filtering out tables that will not be used in caches will speed up the snapshotting process.

      # -- Static environment variables to be applied to ReadySet server containers.
extraEnvironmentVars:

  REPLICATION_TABLES: "<schmema>.<table>,<schmema>.<table>"

      To replicate all tables in a schema, use <schema>.*.

      # -- Static environment variables to be applied to ReadySet server containers.
extraEnvironmentVars:

  REPLICATION_TABLES: "<database>.<table>,<database>.<table>"

Step 5. Start ReadySet

In this step, you'll use the Helm package manager to deploy ReadySet into your EKS cluster.

  1. Install the Helm client.

  2. Use the ReadySet Helm chart to deploy ReadySet to your EKS cluster:

    helm install readyset . --values values.yaml

  3. Confirm that the ReadySet deployment completed successfully, with the pods for the ReadySet Adapter, ReadySet Server, and Consul showing Running under STATUS:

    kubectl get pods -o wide

    NAME                                        READY   STATUS    RESTARTS   AGE   IP               NODE                             NOMINATED NODE   READINESS GATES
readyset-consul-server-0                    1/1     Running   0          5m    192.168.39.169   ip-192-168-43-246.ec2.internal   <none>           <none>
readyset-readyset-adapter-9dbfb77d9-ml92h   2/2     Running   0          5m    192.168.48.46    ip-192-168-43-246.ec2.internal   <none>           <none>
readyset-readyset-server-0                  2/2     Running   0          5m    192.168.18.133   ip-192-168-18-84.ec2.internal    <none>           <none>

  4. Confirm that the persistent volumes for storing ReadySet's snapshot of your database and for ReadySet state details were created successfully:

    kubectl get pv

    NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                           STORAGECLASS   REASON   AGE
pvc-d792b6a8-35ae-456d-8c92-415e473931dc   10Gi       RWO            Delete           Bound    default/data-default-readyset-consul-server-0   gp2                     5m
pvc-ddf75696-9eb7-4e28-a846-2110e889c8de   250Gi      RWO            Delete           Bound    default/state-readyset-readyset-server-0        gp2                     5m

  5. Confirm that a load balancer service was created successfully:

    kubectl get service/readyset-readyset-adapter


    NAME                        TYPE           CLUSTER-IP      EXTERNAL-IP                                                                    PORT(S)                         AGE
readyset-readyset-adapter   LoadBalancer   10.100.46.222   k8s-default-readyset-3cab417124-2b191c9917ce4d43.elb.us-east-1.amazonaws.com   3306:30336/TCP,5432:30185/TCP   5m
    Do not move on to the next step until an EXTERNAL-IP has been assigned to the load balancer. This may take a few minutes.

Step 6. Check snapshotting

As soon as ReadySet is connected to the database, it starts storing a snapshot of your database tables on disk. This snapshot will be the basis for ReadySet to cache query results, and ReadySet will keep its snapshot and cache up-to-date automatically by listening to the database's replication stream. Queries can be cached in ReadySet only once all tables have finished the initial snapshotting process.

In this step, you'll check the status of the snapshotting process. Snapshotting can take between a few minutes to several hours, depending on the size of your dataset.

  1. In your EKS cluster, create a temporary pod containing the psql client:

    kubectl run rs-postgres-client \
--rm --tty -i \
--restart='Never' \
--image=postgres \
--namespace=default \
--command -- bash

  2. Start psql, replacing the --host placeholder with the external IP of your load balancer, and replacing the other placeholders with your database connection details:

    PGPASSWORD=<password> psql \
--host=<external IP of load balancer> \
--port=<port> \
--username=<username> \
--dbname=<database_name>

    You should now be in the SQL shell.

  3. Use ReadySet's custom SHOW READYSET TABLES command to check the snapshotting status of tables in the database ReadySet is connected to:

    SHOW READYSET TABLES;

             table            |    status
------------------------------------------
`public`.`title_basics`   | Snapshotting
`public`.`title_ratings`  | Snapshotted
`public`.`title_episodes` | Not Replicated
(3 rows)

    There are 3 possible statuses:

    • Snapshotting: The initial snapshot of the table is in progress.
    • Snapshotted: The initial snapshot of the table is complete. ReadySet is replicating changes to the table via the database's replication stream.
    • Not Replicated: The table has not been snapshotted by ReadySet. This can be because ReadySet encountered an error (e.g., due to unsupported data types) or the table has been intentionally excluded from snapshotting (via the --replication-tables option).

    Info

    You can start caching queries in ReadySet only once all tables with the Snapshotting status have finished snapshotting and show the Snapshotted status.

  4. If you'd like to track snapshotting progress in greater detail, exit the temporary pod, and then check the ReadySet logs:

    export SERVER=$(kubectl get pods | grep readyset-server | cut -d' ' -f1);

    kubectl logs ${SERVER} -c readyset-server | grep 'Snapshotting table'

    Note

    For each table, you'll see the progress and the estimated time remaining in the log messages (e.g., progress=84.13% estimate=00:00:23).

    2022-12-13T16:02:48.142605Z  INFO Snapshotting table{table=`public`.`title_basics`}: replicators::postgres_connector::snapshot: Snapshotting table context=LogContext({"deployment": "readyset-helm-test"})
2022-12-13T16:02:48.202895Z  INFO Snapshotting table{table=`public`.`title_ratings`}: replicators::postgres_connector::snapshot: Snapshotting table context=LogContext({"deployment": "readyset-helm-test"})
2022-12-13T16:02:48.357445Z  INFO Snapshotting table{table=`public`.`title_ratings`}: replicators::postgres_connector::snapshot: Snapshotting started context=LogContext({"deployment": "readyset-helm-test"}) rows=1246402
2022-12-13T16:02:48.921839Z  INFO Snapshotting table{table=`public`.`title_basics`}: replicators::postgres_connector::snapshot: Snapshotting started context=LogContext({"deployment": "readyset-helm-test"}) rows=5159701
2022-12-13T16:03:11.155418Z  INFO Snapshotting table{table=`public`.`title_ratings`}: replicators::postgres_connector::snapshot: Snapshotting finished context=LogContext({"deployment": "readyset-helm-test"}) rows_replicated=1246402
2022-12-13T16:03:19.927790Z  INFO Snapshotting table{table=`public`.`title_basics`}: replicators::postgres_connector::snapshot: Snapshotting progress context=LogContext({"deployment": "readyset-helm-test"}) rows_replicated=1126400 progress=21.83% estimate=00:01:51
...

    Tip

    To follow the full ReadySet Server logs, use:

    export SERVER=$(kubectl get pods | grep readyset-server | cut -d' ' -f1);

    kubectl logs ${SERVER} -c readyset-server -f

    To follow the ReadySet Adapter logs, use:

    export ADAPTER=$(kubectl get pods | grep readyset-adapter | cut -d' ' -f1);

    kubectl logs ${ADAPTER} -c readyset-adapter -f

  1. In your EKS cluster, create a temporary pod containing the mysql client:

    kubectl run rs-mysql-client \
--rm --tty -i \
--restart='Never' \
--image=docker.io/bitnami/mysql:8.0.30-debian-11-r6 \
--namespace=default \
--command -- bash

  2. Start mysql, replacing the --host placeholder with the external IP of your load balancer, and replacing the other placeholders with your database connection details:

    mysql \
--host=<external IP of load balancer> \
--port=<port> \
--user=<username> \
--password=<password> \
--database=<database_name>

    You should now be in the SQL shell.

  3. Use ReadySet's custom SHOW READYSET TABLES command to check the snapshotting status of tables in the database ReadySet is connected to:

    SHOW READYSET TABLES;

             table            |    status
------------------------------------------
`public`.`title_basics`   | Snapshotting
`public`.`title_ratings`  | Snapshotted
`public`.`title_episodes` | Not Replicated
(3 rows)

    There are 3 possible statuses:

    • Snapshotting: The initial snapshot of the table is in progress.
    • Snapshotted: The initial snapshot of the table is complete. ReadySet is replicating changes to the table via the database's replication stream.
    • Not Replicated: The table has not been snapshotted by ReadySet. This can be because ReadySet encountered an error (e.g., due to unsupported data types) or the table has been intentionally excluded from snapshotting (via the --replication-tables option).

    Info

    You can start caching queries in ReadySet only once all tables with the Snapshotting status have finished snapshotting and show the Snapshotted status.

  4. If you'd like to track snapshotting progress in greater detail, exit the temporary pod, and then check the ReadySet logs:

    export SERVER=$(kubectl get pods | grep readyset-server | cut -d' ' -f1);

    kubectl logs ${SERVER} -c readyset-server | grep 'taking database snapshot'

    Note

    For each table, you'll see the progress and the estimated time remaining in the log messages (e.g., progress=84.13% estimate=00:00:23).

    2022-10-18T17:18:01.685613Z  INFO taking database snapshot: replicators::noria_adapter: Starting snapshot
2022-10-18T17:18:01.803163Z  INFO taking database snapshot:replicating table: replicators::mysql_connector::snapshot: Acquiring read lock table=`readyset`.`users`
2022-10-18T17:18:01.807475Z  INFO taking database snapshot:replicating table: replicators::mysql_connector::snapshot: Replicating table table=`readyset`.`users`
2022-10-18T17:18:01.809739Z  INFO taking database snapshot:replicating table: replicators::mysql_connector::snapshot: Read lock released table=`readyset`.`users`
2022-10-18T17:18:01.810049Z  INFO taking database snapshot:replicating table: replicators::mysql_connector::snapshot: Acquiring read lock table=`readyset`.`posts`
2022-10-18T17:18:01.816496Z  INFO taking database snapshot:replicating table: replicators::mysql_connector::snapshot: Replicating table table=`readyset`.`posts`
2022-10-18T17:18:01.818721Z  INFO taking database snapshot:replicating table: replicators::mysql_connector::snapshot: Read lock released table=`readyset`.`posts`
2022-10-18T17:18:01.822144Z  INFO taking database snapshot:replicating table: replicators::mysql_connector::snapshot: Replication started rows=4990 table=`readyset`.`users`
2022-10-18T17:18:01.822376Z  INFO taking database snapshot:replicating table: replicators::mysql_connector::snapshot: Replication started rows=5000 table=`readyset`.`posts`
2022-10-18T17:18:01.863220Z  INFO taking database snapshot:replicating table: replicators::mysql_connector::snapshot: Replication finished rows_replicated=4990 table=`readyset`.`users`
2022-10-18T17:18:01.864316Z  INFO taking database snapshot:replicating table: replicators::mysql_connector::snapshot: Replication finished rows_replicated=5000 table=`readyset`.`posts`
2022-10-18T17:18:01.966256Z  INFO taking database snapshot: replicators::noria_adapter: Snapshot finished

    Tip

    To follow the full ReadySet Server logs, use:

    export SERVER=$(kubectl get pods | grep readyset-server | cut -d' ' -f1);

    kubectl logs ${SERVER} -c readyset-server -f

    To follow the ReadySet Adapter logs, use:

    export ADAPTER=$(kubectl get pods | grep readyset-adapter | cut -d' ' -f1);

    kubectl logs ${ADAPTER} -c readyset-adapter -f

Next steps

  • Set up monitoring

    The ReadySet Server and ReadySet Adapter export granular time series metrics at <adapter IP or host>:6033/prometheus> and <server IP or host>:6034/prometheus, respectively. The metrics are formatted for easy integration with Prometheus, an open source tool you can use to for storing, aggregating, and querying time series data. You can use this data to, for example, profile SQL query latencies and identify queries to cache with ReadySet.

    Viewing Prometheus metrics

    To view the Prometheus metrics exported by the ReadySet Adapter:

    1. Get the IP of the ReadySet Adapter pod:

      export ADAPTER=$(kubectl get pods | grep readyset-adapter | cut -d' ' -f1);

      kubectl get pods \
--field-selector=metadata.name=${ADAPTER} \
-o=jsonpath='{range.items[*]}{.metadata.item} {.status.podIP} {"\n"}{end}'

    2. Create a temporary pod containing the curl command:

      kubectl run mycurlpod \
--rm --tty -i \
--restart='Never' \
--image=curlimages/curl -- sh

    3. Make the GET requests to the Prometheus endpoint:

      curl -X GET <IP of ReadySet Adapter pod>:6033/prometheus

    To view the Prometheus metrics exported by the ReadySet Server:

    1. Get the IP of the ReadySet Server pod:

      export SERVER=$(kubectl get pods | grep readyset-server | cut -d' ' -f1);

      kubectl get pods \
--field-selector=metadata.name=${SERVER} \
-o=jsonpath='{range.items[*]}{.metadata.item} {.status.podIP} {"\n"}{end}'

    2. Create a temporary pod containing the curl command:

      kubectl run mycurlpod \
--rm --tty -i \
--restart='Never' \
--image=curlimages/curl -- sh

    3. Make the GET requests to the Prometheus endpoint:

      curl -X GET <IP of ReadySet Server pod>:6033/prometheus

  • Cache queries

    Once you've identified queries to cache, use ReadySet's custom SQL commands to check if ReadySet supports them and then cache them in ReadySet.

    Note

    To successfully cache the results of a query, ReadySet must support the SQL features and syntax in the query. For more details, see SQL Support. If an unsupported feature is important to your use case, submit a feature request.