Skip to content

Releases: readysettech/readyset

stable-240328

28 Mar 18:15
Compare
Choose a tag to compare

Release Notes

ReadySet is currently released as a docker container. See the Install with Docker documentation for instructions. It can also be installed from an x86_64/amd64 deb package for Ubuntu 22.04 and Debian 12. See package installation instructions.

What's New

  • ca9856b: Add support for MySQL COM_RESET_CONNECTION command.
  • 7c8a1b7: Add support for MySQL COM_CHANGE_USER command.
  • 22f8532: Readyset now streams proxied results from postgres for queries using the simple query protocol similarly to how it already does for the extended query protocol. This vastly reduces the memory consumption for text-protocol queries as well as improves performance of them in many cases.
  • 7a14b00: Support timestamps in prepared statements from the Java PG driver, pgjdbc.

Fixes

  • a997647: Fixed an issue where range queries returned incorrect results in certain situations
  • f054ca6: Fixed issue on SHOW READYSET STATUS reporting the wrong status when reconnecting to the database.
  • f857866: Fixed a bug where we didn't properly parse type alias following a numeric type cast.

stable-240229

29 Feb 17:24
Compare
Choose a tag to compare

Release Notes

ReadySet is currently released as a docker container. See the Install with Docker documentation for instructions. It can also be installed from an x86_64/amd64 deb package for Ubuntu 22.04 and Debian 12. See package installation instructions.

What's New

  • 048fd4b: Added support for the autoparameterization of range queries
  • a67c2f6: Added support for the autoparameterization of negative numbers
  • ecc1820: Added the cache name as a label on the metrics Readyset emits for queries against caches
  • 47ae35e: Added a feature flag to reduce the number of expensive domain metrics Readyset emits in production deployments
  • 7a00dbb: Readyset now supports the DEALLOCATE SQL statement for both postgres and mysql.
  • d571afd: Removed labels from certain metrics that had very high cardinality to reduce the cost of ingesting and plotting metrics
  • f0d1228: Simplified the metrics Readyset emits to keep track of internal state size
  • 511d6bc: Added a new cache_name label to various metrics to improve debuggability
  • b08683d: Simplified the metrics Readyset emits to count the number of times it starts up
  • 6c3809b: Removed an expensive domain-related metric that duplicated pre-existing metrics
  • b3c0d46: REPLICATION_SERVER_ID can now be configured to use alphanumeric characters for postgres.

Fixes

  • 301f78f: Fixed an issue where Readyset was not correctly handling entries in the GROUP BY clause that identified columns by their position in the column list of the query
  • 8b4ae75: Fixed an issue where Readyset evaluated certain expressions incorrectly
  • a36d8da: Fixed an issue where caches were not queryable after restarts in certain cases
  • 7bc9b5b: Fixed an issue where query log metrics were not being emitted for certain proxied queries
  • 0d4d2ff: Fixed an issue where the wrong metric was being emitted to count the number of misses during a replay.

stable-2401176

18 Jan 21:57
Compare
Choose a tag to compare

Release Notes

ReadySet is currently released as a docker container. See the Install with Docker documentation for instructions. It can also be installed from an x86_64/amd64 deb package for Ubuntu 22.04 and Debian 12. See package installation instructions.

What's New

  • 3476510: ReadySet now allows you to configure a LOG_PATH and LOG_ROTATION. Logs will be written to files prefixed with readyset.log in the log path if set and will be rotated according to the log rotation that is specified (or rotated daily if none is specified).
  • 23a4b84: Optimize SELECT statement parsing by eliminating deuplicated (and thrown away) work.
  • dc16282: Added the ability for the SHOW CACHES SQL extension to display an up-to-date list of caches, even after a restart
  • 9b3b6ac: Added an EXPLAIN CREATE CACHE SQL extension that can be used to check if a given query is supported without proxying the query through ReadySet or creating a new cache.
  • 5ad48a3: Added a convenient interactive script to run a ReadySet demo with. Check it out with bash -c "$(curl -sSL https://launch.readyset.io)"

Fixes

  • 278bf2d: Fix small reporting error in SHOW CACHES and SHOW PROXIED QUERIES where the query execute count could be one of several values if the query was run as an ad hoc query as well as a prepared statement.
  • 3d8c83e: Fixed an issue where caches sometimes weren't being recreated correctly after backwards-incompatible upgrades

beta-2023-11-29

29 Nov 19:36
Compare
Choose a tag to compare

Release Notes

ReadySet is currently released as a docker container. See the Install with Docker documentation for instructions.

What's New

  • 3679fb7: ReadySet will now automatically restore caches even if there is a backwards-incompatible upgrade that invalidates the persisted dataflow graph.

Fixes

  • 66f2d9f: Fixed an issue where ReadySet failed to start up if the instance was passed a --replication-server-id.
  • fc15d1b: Fixed a race condition that could result in a deadlock.
  • c67bb95: Fixed an issue where ReadySet didn't start up correctly if its replication slot was dropped while it was down.

beta-2023-10-26

26 Oct 20:09
Compare
Choose a tag to compare

Release Notes

ReadySet is currently released as a docker container. See the Install with Docker documentation for instructions.

What's New

  • 8b81da2: Added domain packet queueing metrics
  • b67b44b: Improved write performance during replication by writing data to disk periodically in the background.
  • a3f88c5: Added a config option to allow users to configure the interval upon which we report our current position to Postgres.
  • 24bdd0c: Remove support for straddled joins (joins with partial keys which trace to each side of the join), which can sometimes be slow to execute.
  • 7e68db8: Added support for queries which have non equi-join-key filters in left joins, as long as those filters mention columns from only one side of the join.
  • 9b2fc49: Support correlated subqueries on the right-hand side of IN and NOT IN in the column list.
  • 4ee8dba: Replace latency histograms in SHOW PROXIED QUERIES and SHOW CACHES with a simple counter.
  • 456eb17: Limited the number of queries returned from SHOW PROXIED QUERIES to 100,000.
  • e2aac68: Added metric for /metrics payload size.
  • 99e6fb8: Added memory allocator metrics.

Fixes

  • fb6da0f: Fixed an issue where ReadySet failed to shut down in certain situations.
  • d5e0ef5: ReadySet now correctly closes upstream prepared statements when the client requests prepared statements to be closed over either the MySQL or PostgreSQL protocols.
  • 377df52: Fixed an issue where ReadySet would panic if it received an EOF packet from a mysql upstream that had gone away.

beta-2023-09-28

29 Sep 18:19
Compare
Choose a tag to compare

Release Notes

ReadySet is currently released as a docker container. See the Install with Docker documentation for instructions.

What's New

  • 847c0cd: Add support for correlated and uncorrelated NOT EXISTS in the WHERE clause of queries.
  • c948a59: Add a new FOR CACHE <q> clause to the end of the EXPLAIN GRAPHVIZ SQL command to allow limiting the generated graphviz to only the nodes for a named cached query.
  • 9496016: In SHOW READYSET STATUS, ReadySet will display the number of queries that failed to parse if --prometheus-metrics is enabled.
  • e6afd5f: ReadySet will display the process start time in SHOW READYSET STATUS if --prometheus-metrics is enabled.
  • 3d6800b: ReadySet will display latency quantiles in SHOW PROXIED QUERIES and SHOW CACHES if --prometheus-metrics is enabled.
  • 4bb7a17: Displays the last error encountered by the replicator in SHOW READYSET STATUS if the error prevents the replicator from entering the main replication loop. The error is cleared once replication is able to begin.
  • d723221: Add a --no-color flag to the readyset and readyset-server binaries, which allows disabling colored output in logs
  • 309bf55: You can now optionally display only supported proxied queries with the command SHOW PROXIED SUPPORTED QUERIES. Thanks, @altmannmarcelo!
  • dafad7a: Adds a status field for the connection between the ReadySet adapter and the upstream database to SHOW READYSET STATUS.
  • 0f66130: Added a --replication-tables-ignore flag that allows you to replicate all tables other than the ones explicity ignored. Thanks, @altmannmarcelo!
  • 1ce1282: Adds a SHOW READYSET MIGRATION STATUS <id> SQL extension to query the status of a migration initiated via CREATE CACHE CONCURRENTLY. This command will return the status of the migration, which is either "Completed", "Pending", or "Failed with error: ". Executing this statement with an id that has already returned a non-pending status (Completed or Failed) will lead to undefined behavior.
  • 81340d8: Adds an optional CONCURRENTLY keyword to the CREATE CACHE syntax which triggers a non-blocking migration. If the keyword is present, the CREATE CACHE statement will return an id can be used to query the state of the migration via the Controller's /migration_status endpoint. The new CREATE CACHE syntax is CREATE CACHE [CONCURRENTLY] [ALWAYS] [<cache_name>] FROM {<select_statement> | <query_id>}
  • a23caf3: In SHOW READYSET TABLES, ReadySet will display a description column as to why a table is not replicated.

Fixes

  • 864e5a2: Fix proxying of Postgres queries that use the Postgres text protocol to send query parameters when those parameters use types that aren't natively supported by ReadySet.
  • 51b7d21: ReadySet now correctly returns metadata about the table and column for queries that reference columns from tables directly. This allows client libraries that use that metadata to know which columns come from which tables in a query, either proxied or cached.
  • 12a09b5: Fixed an issue where ReadySet got stuck in a retry loop if the upstream's WAL rotated.
  • 4c5d2cf: Fixed an issue where ReadySet's replication slot could grow without bound in the absence of writes to its upstream database.
  • 46a1120: Fixed an issue where we were incorrectly assuming that LSNs in replication events from Postgres would always increase monotonically.

beta-2023-08-23

24 Aug 17:53
Compare
Choose a tag to compare

Release Notes

ReadySet is currently released as a docker container. See the Install with Docker documentation for instructions.

What's New

  • e166f17: Provided a better error message and guidance for passwords that include special characters, which must be percent-encoded.
  • c92db77: Removed unused and hidden fallback_cache feature.
    ec5ee70: Removes references to arguments that are either not well support or not recommended for general use from the output of --help.
  • f245f4c: DROP CACHE now indicates whether a cache was removed via a DELETE result with a row count of either 1 or 0.
  • 1a97b33: By default, ReadySet now does not support creating fully materialized nodes. There is a flag --allow-full-materialization that allows this behavior to be configured, which replaces the previous --forbid-full-materialization when this behavior was reversed. This default prevents issues when a full materialization would have needed to allocate more memory than the system has available.
  • 07ef2a8: Add support for queries with a IN and NOT IN expressions with subqueries on the right-hand side in the projected field list
  • 3c87547: Add support for queries with a WHERE clause containing NOT IN with subqueries on the right-hand side
  • 380b258: Add support for queries with a WHERE clause containing IN with subqueries on the right-hand side
  • 66b4a54: --query-log and --query-log-ad-hoc are now enabled by default if --prometheus-metrics is set, which effectively enables all query metrics by default if metrics are enabled.
  • 70756bf: The deployment name is no longer a required argument for the ReadySet adapter or server, and now defaults to "tmp-readyset". Users should take care to set the same deployment ID for all nodes in a deployment.
  • 0491795: Changes the default listening address of a ReadySet adapter to 127.0.0.1:3307 or 127.0.0.1:5433 when connected to a MySQL or Postgres upstream database respectively.
  • adfd2fc: Renamed NORIA_MEMORY_BYTES to READYSET_MEMORY_LIMIT (and --memory to --memory-limit, similarly), and updated the documentation provided by --help to be more accurate regarding the function of the parameter.

Performance Improvements

  • e662a9d: Improved performance of Postgres cached query response encoding by avoiding an unnecessary copy of the results in memory.

Bug Fixes

  • 4c07dc1: Fixed an issue where ReadySet could crash or return incorrect data in certain situations when multiple rows shared the same value in a join column for a cached query.
  • 854aabb: Fixed an issue with Postgres replication that could cause dropped replication events if ReadySet fell behind while replicating
  • 9b64d46: Fix a bug that led to table replication failures on Postgres when we tried to replicate temporal data that contained leap days in years prior to the year 1 CE.
  • f16dfb0: Fixed a recent regression which broke support for OID types in proxied Postgres queries.
  • fcbeb08: Renames the OUTPUTS_POLLING_INTERVAL env variable to VIEWS_POLLING_INTERVAL. This renaming was missed when the option was renamed to --views-polling-interval.

beta-2023-07-26

27 Jul 18:25
Compare
Choose a tag to compare

2023-07-26 Release Notes

ReadySet is currently released as a docker container. See the Quickstart documentation for instructions on using it.

What's New

  • Add support for more complex conditions in the ON clause of INNER JOINs (504b5c9)
  • Added a mechanism to inspect per-thread memory usage. (f1bbc02)

Performance Improvements

  • Significantly reduce maximum resident memory usage for fully materialized queries by streaming records out of base tables (de3231f)
  • Optimize message flushing in PG extended query protocol implementation (0f56f97)
  • Improve Postgres proxy performance by eliminating an intermediate conversion step (396f41c)

Bug Fixes

  • Fixed a bug which would prevent ReadySet from being able to cache certain queries which perform aggregates over subqueries (a53ab66)
  • Fix an issue where certain queries which use the OR operator between conditions in the WHERE clause could sometimes return the wrong results (f7ffbe6)
  • Fixed a Postgres regression that could trigger a resnapshotting loop when tables were dropped shortly after altering the database schema. (#106)
  • Fixed a Postgres issue that could cause errors when executing prepared statements that referenced enum values that were added to existing enum types.
  • Fixed a similar Postgres issue that triggered on arrays of enum values.

beta-2023-06-29

30 Jun 20:18
Compare
Choose a tag to compare

2023-06-29 Release Notes

Included in this release are binaries for the following platforms:

  • x86_64-unknown-linux-gnu / Amazon Linux
  • aarch64-apple-darwin

What's New

  • Helm chart improvements. [c6ded74, cc1cf3f]
  • The custom SHOW READYSET STATUS SQL statement now includes the minimum and maximum replication offset. da50579
  • Apache Zookeeper is no longer supported. 83ced41
  • Reduced the time ReadySet waits before retrying after a replication error from 30s to 1s. 2649e2f
  • SQL support:
    • Allow spaces between and around arguments to aggregate functions. 529ba86
    • MySQL:
      • Allow arbitrary expressions as arguments to GROUP_CONCAT. 6f16afe
      • Support bit literals (already supported for Postgres). 071678b (thanks @cameronbraid!)
    • Postgres:
      • Support COMMENT statements. a2e9302
  • Added the custom EXPLAIN DOMAINS SQL statement, which gives a list of domain replica shards and the URLs of the workers they're running on, sorted by the worker URL. [8e402e7, 12090f8]

Performance Improvements

  • RocksDB-based persistent storage now uses the block-based SST format, improving post-snapshot compaction speed by 10x for large tables (100+ GiB), and improving get/put performance by 7-28% across current benchmarks. [8a32b30a, d0bec66]

Bug Fixes

beta-2023-05-31

01 Jun 17:39
Compare
Choose a tag to compare

2023-05-31 Release Notes

What's New

  • Revamped helm chart with changes to align with best practices and support
    multiple cloud platforms. a51ac40 b739acb 931d0b9 c4ff836
  • Created the proptest-stateful
    crate and added extensive stateful property tests.
  • ReadySet can now infer the database_type from an upstream_db_url,
    making the --database-type argument optional. If --database-type is
    provided with an upstream_db_url, readyset will validate the database
    type. 8ad4482
  • Improve error messages for why something is unsupported when running
    without an upstream database. 41f3532
  • Add --controller-address option that exports prometheus metrics for the
    readyset-server when running in standalone mode. 688ab93
  • Add replication-server-id flag to allow multiple readyset deployments to
    connect to the same upstream db. Each deployment must have a unique
    replication server id. 896b423
  • Add logging that uses row estimates to report snapshotting progress.
    9f85776
  • Added an experimental feature to automatically inline
    placeholders in a query with literal values when the placeholders are
    unsupported, so that the query can be run against ReadySet. This
    feature is enabled with the --experimental-placeholder-inlining flag.
    This feature is experimental as it could degrade the performance of
    the ReadySet instance if too many inlined instances of a query are
    created. It is recommended that you drop the cache of any query with
    inlined placeholders with high cardinality. 4a31acd
  • Add support for INT2, INT4, and INT8 postgres type aliases. e9518f1

Performance Improvements

  • Controller requests now run asynchronously. 120fb30
  • RocksDB compaction is now done as a background process. 51d5a4c
  • Tuned the retry behavior of extend_recipe. 1b77183
  • Initialize persistent state in a different thread. c78bd61
  • Numerous miscelaneous query performance optimizations.

Bug Fixes

  • Update metrics-rs crate to 0.21 to fix a bug in summary
    calculations
    where they
    were not being calculated over sliding windows of
    time
    .
    1557db0
  • Fixed a bug where DDL actions were lost when we encountered a particular
    error in replication. 1101215
  • Fixed an issue where we returned an empty value instead of BEGIN for
    certain transactional queries. fd08ff1
  • Fixed an incorect timezone conversion that prevented lookups into a date
    field using timestamps that had the same date. 33990c7
  • Fixed a bug with optimized string serialization. 5dec2e1
  • Fixed a panic if offset is longer than result set, specifically in the case
    that a query, has ORDER BY, LIMIT, and OFFSET, has aggregates with a
    group by, and doesn't filter by a primary or unique column. 5d9c5ce
  • Fixed a panic that could happen if a connection failed during a migration.
    020f58a
  • Fixed a bug where the ordering of project and distinct nodes would return
    incorrect results. 236ed82
  • Properly mark PostgreSQL partitioned tables as unsupported. 2abffe1
  • Fix handling of a partial replay where a source node has been dropped.
    94be3fc
  • Properly handle creating tables with FKs. f10319e
  • Properly handle replication offsets for uninitialized base tables. 582b68f