1.5. 3.0.x Branch¶
1.5.1. Upgrade Notes¶
#2228: The default maximum document size has been reduced to 8MB. This means that databases with larger documents will not be able to replicate into CouchDB 3.0 correctly without modification. This change has been made in preparation for anticipated hard upper limits on document size imposed by CouchDB 4.0. For 3.x, the max document size setting can be relaxed via the
[couchdb] max_document_sizeconfig setting.#2228: The default database sharding factor
qhas been reduced to 2 by default. This, combined with automated database resharding (see below), is a better starting place for new CouchDB databases. As in CouchDB 2.x, specify?q=#to change the value upon database creation if desired. The default can be changed via the config[cluster] qsetting.#1523, #2092, #2336, #2475: The “node-local” HTTP interface, by default exposed on port 5986, has been removed. All functionality previously available at that port is now available on the main, clustered interface (by default, port 5984). Examples:
GET /_node/{nodename}/_stats GET /_node/{nodename}/_system GET /_node/{nodename}/_all_dbs GET /_node/{nodename}/_uuids GET /_node/{nodename}/_config GET /_node/{nodename}/_config/couchdb/uuid POST /_node/{nodename}_config/_reload GET /_node/{nodename}/_nodes/_changes?include_docs=true PUT /_node/{nodename}/_dbs/{dbname} POST /_node/{nodename}/_restart GET /_node/{nodename}/{db-shard} GET /_node/{nodename}/{db-shard}/{doc} GET /_node/{nodename}/{db-shard}/{ddoc}/_info
…and so on. Documentation has been updated to reflect this change.
Warning
The _node endpoint is for adminstrative purposes it is NOT intended as an alternative to the regular endpoints (“GET /dbname”, “PUT /dbname/docid” and so on)
#2389: CouchDB 3.0 now requires a server admin user to be defined at startup, or will print an error message and exit. If you do not have one, be sure to create an admin user. (The Admin Party is now over.)
CC-BY-NC 2.0: hehaden @ Flickr¶
#2576: CouchDB 3.0 now requires admin-level access for the
/_all_dbsendpoint.#2339: All databases are now created by default as admin-only. That is, the default new database
_securityobject is now:{ "members" : { "roles" : [ "_admin" ] }, "admins" : { "roles" : [ "_admin" ] } }
This can be changed after database creation.
Due to code changes in #2324, it is not possible to upgrade transparently from CouchDB 1.x to 3.x. In addition, the
couchuputility has been removed from CouchDB 3.0 by #2399. If you are upgrading from CouchDB 1.x, you must first upgrade to CouchDB 2.3.1 to convert your database and indexes, usingcouchupif desired. You can then upgrade to CouchDB 3.0. Or, you can start a new CouchDB 3.0 installation and replicate directly from 1.x to 3.0.#1833, #2358, #1871, #1857: CouchDB 3.0 supports running only under the following Erlang/OTP versions:
19.x - “soft” support only. No longer tested, but should work.
20.x - must be newer than 20.3.8.11 (20.0, 20.1, 20.2 versions all invalid)
21.x - for 21.2, must be newer than 21.2.3
22.x - for 22.0, must be newer than 22.0.5
#1804: By default, views are limited to return a maximum of 2**28 (268435456) results. This limit can be configured separately for views and partitioned views via the
query_limitandpartition_query_limitvalues in the ini file[query_server_config]section.After upgrading all nodes in a cluster to 3.0, add
[rexi] use_kill_all = truetolocal.inito save some intra-cluster network bandwidth.
1.5.1.1. Deprecated feature removal¶
The following features, deprecated in CouchDB 2.x, have been removed or replaced in CouchDB 3.0:
#2089, #2128, #2251: Local endpoints for replication targets, which never functioned as expected in CouchDB 2.x, have been completely removed. When replicating databases, always specify a full URL for the source and target. In addition, the node local
_replicatordatabase is no longer automatically created.#2163: The
disk_sizeanddata_sizefields have been retired from the database info object returned byGET /{db}/. These were deprecated in CouchDB 2.x and replaced by thesizesobject, which contains the improvedfile,activeandexternalsize metrics. Fauxton has been updated to match.#2173: The ability to submit multiple queries against a view using the POST to
/{db}/_design/{ddoc}/_view/{view}with the?queries=option has been replaced by the new queries endpoint. The same is true of the _all_docs, _design_docs, and _local_docs endpoints. Specify akeysobject when POST-ing to these endpoints.#2248: CouchDB externals (
_external/) have been removed entirely.#2208: CouchDB no longer supports the
delayed_commitsoption in the configuration file. All writes are now full commits. The/_ensure_full_commitAPI endpoint has been retained (as a no-op) for backwards compatibility with old CouchDB replicators.#2395: The security object in the
_usersdatabase cannot be edited by default. A setting exists in the configuration file to revert this behaviour. The ability to override the disable setting is expected to be removed in CouchDB 4.0.
1.5.1.2. Deprecated feature warnings¶
The following features are deprecated in CouchDB 3.0 and will be removed in CouchDB 4.0:
Show functions (
/{db}/{ddoc}/_show)List functions (
/{db}/{ddoc}/_list)Update functions (
/{db}/{ddoc}/_update)Virtual hosts and ini-file rewrites
Rewrite functions (
/{db}/{ddoc}/_rewrite)
1.5.2. Version 3.0.1¶
1.5.2.1. Features and Enhancements¶
Fauxton was updated to version v1.2.3.
1.5.2.2. Bugfixes¶
#2441: A memory leak when encoding large binary content was patched. This should resolve a long-standing gradual memory increase bug in CouchDB.
#2613: Simultaneous attempts to create the same new database should no longer result in a 500 Internal Server Error error.
#2678: Defaults for the
smooshcompaction daemon are now consistent with the shippeddefault.inifile.#2680: The Windows CouchDB startup batch file will no longer fail to start CouchDB if incompatible versions of OpenSSL are on the
PATH.#2741: A small performance improvement in the
couch_serverprocess was made.#2745: The
require_valid_userexception logic was corrected.#2643: The
users_db_security_editablesetting is now in the correct section of thedefault.inifile.#2654: Filtered changes feeds that need to rewind partially should no longer rewind all the way to the beginning of the feed.
#2655: When deleting a session cookie, CouchDB should now respect the operator-specified cookie domain, if set.
#2690: Nodes that re-enter a cluster after a database was created (while the node was offline or in maintenance mode) should more correctly handle creating local replicas of that database.
#2805: Mango operators more correctly handle being passed empty arrays.
#2716, #2738: The
remshutility will now try and guess the node name and Erlang cookie of the local installation. It will also respect theCOUCHDB_ARGS_FILEenvironment variable.#2797: The cluster setup workflow now uses the correct logging module.
#2818: Mango now uses a safer method of bookmark creation that prevents unexpectedly creating new Erlang atoms.
#2756: SpiderMonkey 60+ will no longer corrupt UTF-8 strings when various JS functions are applied to them.
Multiple test case improvements, including more ports of JS tests to Elixir.
1.5.3. Version 3.0.0¶
1.5.3.1. Features and Enhancements¶
#1789: User-defined partitioned databases.
These special databases support user-driven placement of documents into the same shard range. JavaScript views and Mango indexes have specific optimizations for partitioned databases as well.
Two tweakable configuration parameters exist:
#1972, #2012: Automated shard splitting. Databases can now be re-sharded while online to increase the
qfactor to a larger number. This can be configured to require specific node and range parameters upon execution.#1910: Automatic background indexing, internally known as
ken. This subsystem ensures secondary indexes (such as JavaScript, Mango, and text search) are kept up to date, without requiring an external query to trigger building them. Many configuration parameters are available.#1904: Completely rewritten automatic compaction daemon, internally known as
smoosh. This subsystem automatically triggers background compaction jobs for both databases and views, based on configurable thresholds.#1889, #2408: New IO Queue subsystem implementation. This is highly configurable and well-documented.
#2436, #2455: CouchDB now regression tests against, and officially supports, running on the
arm64v8(aarch64) andppc64le(ppc64el) machine architectures. Convenience binaries are generated on these architectures for Debian 10.x (“buster”) packages, and for the Docker containers.#1875, #2437, #2423: CouchDB now supports linking against SpiderMonkey 60 or SpiderMonkey 1.8.5. SpiderMonkey 60 provides enhanced support for ES5, ES6, and ES2016+. Full compatibility information is available at the ECMAScript compatibility table. Click on “Show obsolete platforms”, then look for “FF 60 ESR” in the list of engine types.
However, it was discovered that on some ARM 64-bit distributions, SM 60 segfaults frequently, including the SM 60 packages on CentOS 8 and Debian 10.
As a result, CouchDB’s convenience binaries only link against SM 60 on the ``x86_64`` and ``ppc64le`` architectures. This includes the Docker image for these architectures.
At present, CouchDB ships with SM 60 linked in on the following binary distributions:
Debian buster (10.x)
CentOS / RedHat 8.x
macOS (10.10+)
Windows (7+)
Docker (3.0.0)
FreeBSD (CURRENT)
We expect to add SM 60 support to Ubuntu with Focal Fossa (20.04 LTS) when it ships in April 2020.
It is unlikely we will backport SM 60 packages to older versions of Debian, CentOS, RedHat, or Ubuntu.
The Windows installer has many improvements, including:
Prompts for an admin user/password as CouchDB 3.0 requires * Will not overwrite existing credentials if in place
No longer remove user-modified config files, closing #1989 * Also will not overwrite them on install.
Checkbox to disable installation of the Windows service
Friendly link to these online release notes in the exit dialog
Higher resolution icon for HiDPI (500x500)
Warning
Windows 8, 8.1, and 10 require the .NET Framework v3.5 to be installed.
#2037: Dreyfus, the CouchDB side of the Lucene-powered search solution, is now shipped with CouchDB. When one or more Clouseau Java nodes are joined to the cluster, text-based indexes can be enabled in CouchDB. It is recommended to have as many Clouseau nodes as you have CouchDB nodes. Search is advertised in the feature list present at
GET /if configured correctly (#2206). Configuration and installation documentation is available.#2411: The
/_upendpoint no longer requires authentication, even whenrequire_valid_useristrue.#2392: A new
_metricsrole can be given to a user. This allows that user access only to the/_node/{node}/_statsand/_node/{node}/_systemendpoints.#1912: A new alternative
systemd-journald loggingbackend has been added, and can be enabled through the ini file. The new backend does not include CouchDB’s microsecond-accurate timestamps, and uses thesd-daemon(3)logging levels.#2296, #1977: If the configuration file setting
[couchdb]single_nodeis set totrue, CouchDB will automatically create the system databases on startup if they are not present.#2338, #2343: POST request to CouchDB views and the
/{db}/_all_docs,/{db}/_local_docsand/{db}/_design_docsendpoints now support the same functionality as GET. Parameters are passed in the body as a JSON object, rather than in the URL when using POST.#2292: The
_scheduler/docsand_scheduler/infoendpoints now return detailed replication stats for running and pending jobs.#2282, #2272, #2290: CouchDB now supports specifying separate proxies for both the
sourceandtargetin a replication viasource_proxyandtarget_proxykeys. The API documentation has been updated.#2240: Headers are now returned from the
/{db}/_changesfeed immediately, even when there are no changes available. This avoids client blocking.#2005, #2006: The name of any node can now be retrieved through the new API endpoint
GET /_node/{node-name}.#1766: Timeouts for requests,
all_docs, attachments, views, and partitioned view requests can all be specified separately in the ini file under the[fabric]section. Seedefault.inifor more detail.#1963: Metrics are now kept on the number of partition and global view queries, along with the number of timeouts that occur.
#2452, #2221: A new configuration field
[couch_httpd_auth] same_sitehas been added to set the value of the CouchDB auth cookie’sSameSiteattribute. It may be necessary to set this tostrictfor compatibility with future versions of Google Chrome. If CouchDB CORS support is enabled, set this toNone.
1.5.3.2. Performance¶
#2277: The
couch_serverprocess has been highly optimized, supporting significantly more load than before.#2360: It is now possible to make the rexi interface’s unacked message limit configurable. A new, more optimized default (5, lowered from 10) has been set. This results in a ~50% improvement on view queries on large clusters with
q ≥ 8.#2280: Connection sharing for replication now functions correctly when replicating through a forward proxy. Closes #2271.
#2195, #2207: Metrics aggregation now supports CouchDB systems that sleep or hibernate, ensuring that on wakeup does not trigger thousands of unnecessary function calls.
#1795: Avoid calling
fabric:update_docswith empty doc lists.#2497: The setup wizard no longer automatically creates the
_global_changesdatabase, as the majority of users do not need this functionality. This reduces overall CouchDB load.
1.5.3.3. Bugfixes¶
#1752, #2398, #1803: The cluster setup wizard now ensures a consistent UUID and http secret across all nodes in a cluster. CouchDB admin passwords are also synced when the cluster setup wizard is used. This prevents being logged out when using Fauxton as a server admin user through a load balancer.
#2388: A compatibility change has been made to support replication with future databases containing per-document access control fields.
#2379: Any replicator error messages will provide an object in the response, or null, but never a string.
#2244, #2310: CouchDB will no longer send more data than is requested when retrieving partial attachment data blocks.
#2138: Manual operator updates to a database’s shard map will not corrupt additional database properties, such as partitioning values.
#1877: The
_purgeand_purged_infos_limitendpoints are now correctly restricted to server admin only.#1794: The minimum purge sequence value for a database is now gathered without a clustered
_all_docslookup.#2351: A timeout case clause in
fabric_db_infohas been normalised to match other case clauses.#1897: The
/{db}/_bulk_docsendpoint now correctly catches invalid (i.e., non-hexadecimal)_rev_values and responds with a 400 Bad Request error.#2321: CouchDB no longer requires Basic auth credentials to reach the
/_sessionendpoint for login, even whenrequire_valid_useris enabled.#2295: CouchDB no longer marks a job as failed permanently if the internal doc processor crashes.
#2178: View compaction files are now removed on view cleanup.
#2179: The error message logged when CouchDB does not have a
_usersdatabase is now less scary.#2153: CouchDB no longer may return a
badmatcherror when queryingall_docswith a passedkeysarray.#2137: If search is not available, return a 400 Bad Request instead of a 500 Internal Server Error status code.
#2077: Any failed
fsync(2)calls are now correctly raised to avoid data corruption arising from retry attempts.#2027: Handle epoch mismatch when duplicate UUIDs are created through invalid operator intervention.
#2019: If a database is deleted and re-created while internal cluster replication is still active, CouchDB will no longer retry to delete it continuously.
#2003, #2438: CouchDB will no longer automatically reset an index file if any attempt to read its header fails (such as when the couch_file process terminates unexpectedly). CouchDB now also handles the case when a view file lacks a proper header.
#1983: Improve database “external” size calcuation to be more precise.
#1971: Correctly compare ETags using weak comparison methods to support W/ prefix added by some load balancer configurations.
#1901: Invalid revision specified for a document update will no longer result in a
badargcrash.#1845: The
end_timefield in/_replicatenow correctly converts time to UTC.#1824:
rexistream workers are now cleaned up when the coordinator process is killed, such as when the ddoc cache is refreshed.#1770: Invalid database
_securityobjects no longer return afunction_clauseerror and stack trace.#2412: Mango execution stats now correctly count documents read which weren’t followed by a match within a given shard.
#2393, #2143: It is now possible to override the query server environment variables
COUCHDB_QUERY_SERVER_JAVASCRIPTandCOUCHDB_QUERY_SERVER_COFFEESCRIPTwithout overwriting thecouchdb/couchdb.cmdstartup scripts.#2426, #2415: The replicator now better handles the situation where design document writes to the target fail when replicating with non-admin credentials.
#2444, #2413: Replicator error messages are now significantly improved, reducing
function_clauseresponses.#2454: The replication auth session plugin now ignores other cookies it may receive without logging an error.
#2458: Partitioned queries and dreyfus search functions no longer fail if there is a single failed node or rexi worker error.
#1783: Mango text indexes no longer error when given an empty selector or operators with empty arrays.
#2466: Mango text indexes no longer error if the indexed document revision no longer exists in the primary index.
#2486: The
$lt,$lte,$gt, and$gteMango operators are correctly quoted internally when used in conjunction with a text index search.#2493: The
couch_auth_cacheno longer has a runaway condition in which it creates millions of monitors on the_usersdatabase.
1.5.3.4. Other¶
The 3.0.0 release also includes the following minor improvements:
#2472: CouchDB now logs the correct, clustered URI at startup (by default: port
5984.)#2034, #2416: The path to the Fauxton installation can now be specified via the
COUCHDB_FAUXTON_DOCROOTenvironment variable.#2447: Replication stats are both persisted when jobs are re-created, as well as properly handled when bulk document batches are split.
#2410, #2390, #1913: Many metrics were added for Mango use, including counts of unindexed queries, invalid index queries, docs examined that do and don’t meet cluster quorum, query time, etc.
#2152, #2504: CouchDB can now be started via a symlink to the binary on UNIX-based platforms.
#1844: A new internal API has been added to write custom Erlang request-level metrics reporting plugins.
#2293, #1095: The
-args_file,-configand-couch_iniparameters may now be overridden via theCOUCHDB_INI_FILESenvironment variable on UNIX-based systems.#2352: The
remshutility now searches for the Erlang cookie inERL_FLAGSas well asvm.args.#2324: All traces of the (never fully functional) view-based
_changesfeed have been expunged from the code base.#2337: The md5 shim (introduced to support FIPS-compliance) is now used consistently throughout the code base.
#2270: Negative and non-integer
heartbeatvalues now return 400 Bad Request.#2268: When rescheduling jobs, CouchDB now stops sufficient running jobs to make room for the pending jobs.
#2186: CouchDB plugin writers have a new field in which endpoint credentials may be stashed for later use.
#2183:
dev/runnow supports an--extra-argsflag to modify the Erlang runtime environment during development.#2105:
dev/runno longer fails on unexpected remote end connection close during cluster setup.#2118: Improve
couch_epiprocess replacement mechanism using map childspecs functionality in modern Erlang.#2111: When more than
MaxJobsreplication jobs are defined, CouchDB now correctly handles job rotation when some jobs crash.#2020: Fix full ring assertion in fabric stream shard replacements
#1925: Support list for docid when using
couch_db:purge_docs/3.#1642:
io_priorityis now set properly on view update and compaction processes.#1865: Purge now supports >100 document IDs in a single request.
#1861: The
vm.argsfile has improved commentary.#1808: Pass document update type for additional checks in
before_doc_update.#1835: Module lists are no longer hardcoded in
.appfiles.#1798, #1933: Multiple compilation warnings were eliminated.
#1826: The
couch_replicator_managershim has been fully removed.#1820: After restarting CouchDB, JS and Elixir tests now wait up to 30s for it to be ready before timing out.
#1800:
make elixirsupports specifying individual tests to run withtests=.#1805:
dev/runsupports--with-haproxyagain.#1774:
dev/runnow supports more than 3 nodes.#1779: Refactor Elixir test suite initialization.
#1769: The Elixir test suite uses Credo for static analysis.
#1776: All Python code is now formatted using Python black.
#1786:
dev/run: do not create needlessdev/data/directory.#2482: A redundant
get_ring_optscall has been removed fromdreyfus_fabric_search.#2506: CouchDB’s release candidates no longer propagate the RC tags into each Erlang application’s version string.
#2511: recon, the Erlang diagnostic toolkit, has been added to CouchDB’s build process and ships in the release + convenience binaries.
Fauxton updated to v1.2.3, which includes:
Support multiple server-generated warnings when running queries
Partitioned database support
Search index support
Remove references to deprecated dbinfo fields
Improve accessibility for screen readers
Numerous CSS fixes
Improved test cases:
Many, many test race conditions and bugs have been removed (PR list too long to include here!)
More test cases were ported to Elixir, including:
#2431:
chttpd_purge_testshave been improved in light of CI failures.#2432: Address flaky test failure on
t_invalid_view/1.#2363: Elixir tests now run against a single node cluster, in line with the original design of the JavaScript test suite. This is a permanent change.
#1893: Add “w:3” for lots of doc tests.
#1939, #1931: Multiple fixes to improve support in constrained CI environments.
#2346: Big-endian support for the
couch_compresstests.#2314: Do not auto-index when testing
update=falsein Mango.#2141: Fix
couch_viewsencoding test.#2123: Timeout added for
fold_docs-with_different_keystest.#2114: EUnit tests now correctly inherit necessary environment variables.
#2122: :meck.unload() is now called automatically after every test.
#2098: Fix
cpse_test_purge_replicationeunit test.#2084: Increase timeouts on two slow btree tests.
#1922: Tests added for shard splitting functionality.
#1869: New test added for doc reads with etag
If-None-Matchheader.#1831: Re-introduced cpse_test_purge_seqs test.
#1790: Reorganise
couch_flag_config_testsinto a proper suite.#1785: Use
devcleanon elixir target for consistency of Makefile.#2476: For testing,
Triqhas been replaced withPropEras an optional dependency.
External dependency updates:
A llama! OK, no, not really. If you got this far…thank you for reading.