Release Notes

What’s New in Cordra v2.0

Please visit Cordra v2.0 Features for the core features offered by version 2 of the Cordra software.

Below you will find a list of incremental changes made to the software.

Changes in 2.3.0 release (2021-04-01):

  • Endpoint of DOIP API for HTTP clients is /doip; see DOIP API for HTTP Clients.
  • Access tokens for invoking DOIP operations can be managed using operation identifiers 20.DOIP/Op.Auth.Token, 20.DOIP/Op.Auth.Introspect, and 20.DOIP/Op.Auth.Revoke; see Access Token Operations.
  • Operation 0.DOIP/Op.ListOperations and the listMethods API now only list operations/methods for which the calling user has ACL-level permission.
  • Aliases for service and core operation ids in DOIP; see Aliases.
  • New NPM project @cnri/cordra-tool to facilitate development and testing of Cordra configuration of types, operations, and lifecycle hooks. See the README.md in the NPM package for details.
  • Global static methods can be placed on the design object and can be accessed as DOIP custom operations targeting the service object; see Design-level Static Methods.
  • New lifecycle hook customizeQuery allows restricting or augmenting user-supplied queries; see Customize Query Hook.
  • New lifecycle hooks afterCreateOrUpdate and afterDelete; see Lifecycle Hooks.
  • Lifecycle hooks for the Design object and Type objects; see Hooks for the Design Object and Type Objects.
  • The convenience API endpoints /initData, /design, and /schemas now properly apply ACLs for the Design object and Type objects.
  • New query parameters “facets” and “filterQueries”; see Search for objects and Faceted Search.
  • Schema validation now properly validate “format”:”date” and “format”:”time”.
  • The “createHandleValues” lifecycle hook allows access to the “context” parameter, which can be used to inspect “context.effectiveAcl” to determine whether an object is public.
  • The “onObjectResolution” lifecycle hook can now inspect a context property “isSearch” to determine if it is being apply to a standalone object resolution or to a search result.
  • New ACL “payloadReaders” allows further restricting which object readers are authorized to read payloads.
  • Type methods called using GET (or POST with empty body) allow passing input via a query parameter called “params”.
  • Cordra JavaScript comes with polyfills supporting many features up to ECMAScript 2017, to facilitate transpiling next-generation JavaScript source into ES5 for use in Cordra.
  • Cordra schema JavaScript allows more control over the returned server response when throwing errors; see Throwing Errors in Schema JavaScript.
  • The JavaScript cordra-client now throws objects which are instanceof Error for better stack traces.

Changes in 2.2.0 release (2020-10-06):

  • Cordra schemas in Type objects can now reference other Cordra schemas using $ref property; see Schema References.
  • Type methods and lifecycle hooks have access to Cordra object payloads; before only the JSON portion of the digital object was accessible to those methods. See Cordra.js Module.
  • Type methods can now accept and return arbitrary byte streams (instead of just JSON); see Direct I/O.
  • Type methods, for use with the REST API, can specify that they can be invoked using HTTP GET (instead of just POST); see Allowing GET.
  • The onObjectResolution hook can now control access during payload retrievals and the new onPayloadResolution hook can produce bytes or override stored bytes during a payload retrieval; see Lifecycle Hooks.
  • The JavaScript cordra-client now works with Node.js, in addition to the browser.
  • A bug was fixed to ensure the createHandleValues JavaScript hook returns handle records as intended.
  • A bug was fixed to stop memory a leak issue during payload creation and update.
  • A bug was fixed to expose object.metadata to beforeSchemaValidation hook on update.
  • The listMethods call is now available to authorized readers (instead of just writers).
  • Default authorization configuration is changed for new Cordra installs. New installs by default require authentication to read any objects (other than Type/Schema objects and the design object which are public). If you prefer the old default, edit the authorization configuration via the design object or the admin interface. Existing installations after software update will continue to work as before.
  • A new cordra-schema-util library is introduced with utility functions related to Jackson and schema validation, including certain utilities formerly part of cordra-client; developers can leverage this library for performing project-specific checks.
  • Authentication functionality will now use the response from onObjectResolution hook applied to user objects; before only stored user objects were considered.
  • Cordra distribution now includes hdl-convert-key script to facilitate converting keys among Handle, PEM, and JWK formats.
  • cordraUtil.signWithKey, cordraUtil.signWithCordraKey, and cordraUtil.getCordraPublicKey utility methods are added to the embedded JavaScript library for use in Cordra lifecycle hooks and type methods to facilitate the production and checking of signatures.
  • DOIP configuration allows setting serviceName and serviceDescription, which will be included in the response to the Hello request.
  • The SchemaImporter tool will preserve existing JavaScript when updating a Type/Schema object; to delete existing JavaScript during an import procedure, a new –delete-javascript flag can be specified.

Changes in 2.1.0 release (2020-05-08):

  • Allow use of Elasticsearch as Cordra indexer (this did not work correctly in v2.0.0).
  • Fix issue with type method call locking (object passed to method is now acquired inside the lock).
  • Cordra UI object editor will now retain falsy non-required properties either fetched from server or from “Edit As Json”. Falsy non-required properties can be added or removed using “Edit As Json”.
  • Allow type methods to take input other than JSON objects (for example, strings).
  • TokenUsingCordraClient, when a request fails due to a server-invalidated token, will retry with a new token.
  • Add new /search HTTP API for searching; enable search requests to be sent as JSON via POST; see Search for objects.
  • Enable search with “queryJson” parameter to match content of object against supplied JSON; see Search for objects.
  • Enable system properties to control trusted HTTPS certificates when using HttpCordraClient; see TLS Configuration.
  • Update dependencies, notably ZooKeeper, Curator, and Kafka.
  • Facilitate use of TLS with ZooKeeper, Kafka, and other services; see Enabling TLS.
  • Add getOrNull method to cordra-client.js. If the digital object does not exist, get method throws an exception; getOrNull method instead returns null value.
  • Support new configuration options for Solr and Elasticsearch; see Configuring Indexing Backend.
  • Allow limited use of ES6 (that supported by Nashorn) in JavaScript used with Cordra types. Notably “let” and “const” are usable even in Java 8; later Java versions offer partial support for arrow functions, for…of loops, and backtick-delimited strings.
  • Support sending exceptionally long queries to Solr (previously Solr would by default reject queries over seven or eight thousand characters long).
  • Cordra UI now incorporates dependencies that it formerly fetched from CDNs.
  • Built-in handle server now uses the same listenAddress as Cordra’s HTTP interfaces by default.
  • Remove unexpected error.log messaging about DOIP interface when using Cordra in a servlet container.
  • Fix bug preventing use of filesystem bdbje storage with multi storage configuration.
  • Filesystem bdbje storage can have the path to the Cordra data directory overridden by configuration; see Configuring Storage Backend.
  • Custom storage backends can automatically receive the path to the Cordra data directory as property cordraDataDir of their configuration options; see Configuring Storage Backend.
  • Use “exec” in command-line scripts to ensure that only one process is spawned.
  • Copy secure properties of a digital object when creating a version of that object.
  • Prevent simultaneous changes to digital objects when their versions are being created.
  • Make HttpCordraClient.getNewHttpClient and other methods available for overriding in a subclass to change HTTP client configuration.

Warning

Cordra Beta v2.0 versions released after 2018-08-01 include an improved JavaScript API which is incompatible with earlier released versions of Cordra Beta v2.0. If your Cordra configuration includes schema JavaScript, see here for an upgrade path.

Cordra users upgrading from early versions of the Cordra Beta v2.0, who did not use schema JavaScript (apart from the default User schema JavaScript, which will be automatically upgraded if it has not been edited), do not in general need to take any action.

Also, earlier versions of Cordra would return all results to searches with pageSize=0. To restore the former behavior, you can add “useLegacySearchPageSizeZeroReturnsAll”:true to the Cordra Design Object. By default a search with pageSize=0 returns the number of matched objects but no object content.

As of Cordra Beta v2.0 versions released after 2019-06-01, Kafka-based replication no longer includes payloads in the replication messages. If you are using replication and you need payloads to replicate there is a boolean property “includePayloadsInReplicationMessages” that can be set to true on the Design object. Note that the current implementation of replication with “includePayloadsInReplicationMessages” may require special Kafka configuration or may not be suitable when there are large payloads.

Cordra Beta v2.0 versions released after 2019-08-02 only support Elasticsearch version 6 and 7 as indexing backends. If you have an existing Elasticsearch 5 index, you’ll need to upgrade and reindex.

Cordra v2.0.0 and later only support using access tokens to create HTTP REST API sessions. If your application uses the cookie-based Legacy Sessions API, you will need to upgrade to use the new Token API. To restore this former behavior, you can add “useLegacySessionsApi”:true to the Cordra design object. See Access Token API for details on the new API.

As of Cordra v2.0.0, all authentication requests must be made over a secure HTTPS connection. To allow authentication over insecure channels, you can add “allowInsecureAuthentication”:true to the Cordra design object.

Cordra v2.0.0 uses memory sessions by default. If you have a distributed Cordra installation which uses Tomcat session replication, you will need to configure the Cordra session manager to use Tomcat-managed sessions. See Distributed Sessions Manager.

Changes in 2.0.0 release (2019-10-09):

  • Add built-in identifier resolution; see Identifiers and Handle Integration.
  • Ensure Cordra object ids are syntactically valid handles.
  • Make DOIP listener active on port 9000 by default.
  • Add handle values to Cordra object id records for DOIP clients to locate objects.
  • Add generateId JavaScript lifecycle hook; see Generate Object Id Hook.
  • Allow handleReference in schemas to refer to objects of any types, or any types except a fixed list, instead of requiring a fixed list of allowed types.
  • Allow authentication only over HTTPS by default.
  • New options for HTTPS configuration, in particular to allow updating certificate without restart; see Configuring HTTPS Keys.
  • Change default session management to memory sessions; add separate configuration option for using Tomcat-managed sessions. See Distributed Sessions Manager.
  • Add cordraUtil.js module; see CordraUtil.js Module.
  • Prevent MongoDbStorage from storing JSON numbers not representable as MongoDB numbers.
  • Make client-supplied requestContext available to JavaScript hooks; see Request Context.
  • Add parameter filter to search and retrieval APIs to allow returning only parts of the objects specified by JSON pointers.
  • Upgrade dependencies; support Elasticsearch 6 and 7 (but not 5).
  • Add Access Token API and deprecate Legacy Sessions API.
  • Add script to allow easier creation of Handle key pairs.
  • Support providing jar files in data/lib and sub-directories.
  • New API GET /check-credentials to test authentication whether direct or token/session-based.
  • Add batch files, e.g., startup and shutdown, for Windows.
  • Update technical manual significantly.

Changes in 2019-06-12 beta release:

  • Digital Object Hashing, which allows hashes of the object content to automatically be included in object metadata.
  • Fix bug to ensure that any errors resulting from sessions setup (see Distributed Sessions Manager) are visible at startup.
  • Prevent creating a digital object with a zero-character, i.e., empty, identifier. If the use of previous versions of Cordra resulted in digital objects with empty identifiers, you can delete them with this recovery API call: DELETE /objects/?deleteObjectWithEmptyId.
  • Ensure that initial default schemas have appropriate createdOn and modifiedOn metadata.
  • Versions (see Digital Object Versioning) are now immutable by default; they can be made mutable by setting a Design object flag “enableVersionEdits”.
  • Improved Cordra software performance.
  • Fix bug that in rare cases could allow user and group changes to not be immediately visible to the portion of Cordra process that authenticates users.
  • Fixes to migration from Cordra v1.
  • UI fix to prevent issues with schemas containing spaces.
  • Allow configuration of cookies used for Cordra sessions; see Design Object.
  • Kafka-based replication no longer includes payloads in the replication messages. If you are using replication and you need payloads to replicate there is a boolean property “includePayloadsInReplicationMessages” that can be set to true on the Design object. Note that the current implementation of replication with “includePayloadsInReplicationMessages” may require special Kafka configuration or may not be suitable when there are large payloads.
  • Storage modules “custom” and “multi”; see Configuring Storage Backend.
  • To facilitate clients passing contextual information to the storage backend, HTTP API calls admit a query parameter “requestContext”. This will be made available to the instance of StorageChooser used by the “multi” storage module. See Multiple Storages.

Changes in 2019-04-09 beta release:

  • Fix bug which prevented starting additional webapps in data/webapps.
  • Add new config.json property reindexing.async; reindexing.priorityTypes no longer causes async reindexing automatically. See Reindexing.
  • Improve documentation around possible issues reindexing when using types like JavaScriptDirectory.

Changes in 2019-03-29 beta release:

  • Ensure that sources of internal CNRI libraries are included in distribution.
  • Allow minRf to be configured in Solr indexer configuration.
  • Fix client tools ExportByQuery and ImportObjects which can now optionally connect to a MongoDB backend for internal metadata.
  • New server-side tools “export-tool” and “import-tool” which can connect directly to Cordra storage in order to export and import objects; also “ids-by-query” to retrieve a list of ids from a running Cordra. See Import and Export Tools.
  • Improve performance of reindexing under Elasticsearch.
  • Make it so that components of Cordra object “metadata” are indexed under fields with names like “metadata/createdOn”, etc.
  • New MongoDB storage configuration option “maxTimeMsLongRunning”, which defaults to a large value, to prevent processing timeouts on slow reindexing operations.
  • New HTTP API for searches which returns only object ids instead of full objects, using query parameter “&ids”.
  • Fixed bug causing incorrectly sorted search results when using MongoDB storage.
  • Fixed bug causing metadata “createdOn” and “modifiedOn” to differ for a newly created object.

Changes in 2019-03-09 beta release:

  • Substantial changes to UI.
  • Configurable session management backend; see Distributed Sessions Manager.
  • User schemas can include flags to activate/deactivate users; see “auth”: “accountActive”.
  • Single-instance Cordra installation allows additional jar files to be made available to Cordra by placing in data/lib directory.
  • A file setenv.sh next to startup will be run by startup (for ease of setting environment variables in automatic installations).
  • Remove all internal dependence on objatt_ fields in the index. This allows ignoring those fields in a Solr or Elasticsearch install, if desired to save index disk space.
  • Schemas can indicate that certain fields should not be stored or retrievable plain, but instead stored as a hash and salt which can be validated. Useful for secure tokens. See secureProperty.
  • New API GET /startupStatus to indicate when startup has partially failed; intended to be used in situations where HTTP access to Cordra is much easier than checking logs. See Startup Status API.
  • Upgrade Jetty backend in single-instance install; now supports HTTP/2 in Java 9 or later.
  • /uploadObjects API now should use POST rather than PUT.
  • GET /acls now only requires read permission.

Changes in 2019-01-31 beta release:

  • New objectForIndexing JavaScript hook to adjust how the object is indexed; see Example: Modification of the Indexed Object.
  • Required properties with schema cordra.type.autoGeneratedField were previously populated only if present with some value, even the empty string; now they are auto-generated even if missing.
  • Changed default value of reindexing configuration property batchSize to 16, which allows better performance with the default "lockDuringBackgroundReindex": true.
  • Fixed UI bug which prevented saving objects with missing but not required enum and boolean properties.
  • In the UI, the admin schema editor now allows editing schema JavaScript.
  • In the UI, added and edited schemas are now usable immediately instead of requiring a page refresh.
  • MongoDB storage now allow configuration of databaseName, collectionName, and gridFsBucketName.
  • Fixed bug which could cause schemas to be unknown to Cordra after a reindex in certain configurations.

Changes in 2019-01-11 beta release:

  • Improvements to logging of reindexing, and speed of reindexing when using MongoDB storage.
  • UI fix to prevent possible XSS in use of Toastr to show error messages.
  • Configurable ACLs for schema methods; see Authorization for Type Methods.

Changes in 2018-12-06 beta release:

  • Background reindexing fix to ensure objects are (by default) locked during reindexed; see Reindexing.
  • New /reindexBatch API; see Reindex Batch API.
  • Update documentation for /uploadObjects API.

Changes in 2018-11-27 beta release:

  • General performance improvements.
  • Cordra authorization now allows groups to include other groups. Note: to make use of this feature, existing deployments will need to edit the Group schema to allow the “users” property to include handles of objects of type “Group” as well as type “User”.
  • Reindexing performance improvements and new configuration options; see Reindexing.
  • Ensure “Content-Type: application/json; charset=UTF-8” in more responses.

Version 1.0

  • Version 1.0.7 fixes a sporadic classloading issue experienced rarely by some users.
  • Version 1.0.6 has several minor bugfixes: HTTPS no longer asks for a client-side certificate; Handle resolution is aware of recent GHR changes; and the internal implementation of payload indexing is streamlined.
  • Version 1.0.5 fixes a performance bottleneck in indexing new objects, and also includes the full source needed to build Cordra.
  • Version 1.0.4 adds HTTP Range requests, as well as the “indexPayloads” property to allow turning off indexing of payloads.
  • Version 1.0.3 changes how payloads are associated with Cordra objects. Now any Cordra object can be associated with zero or more named payloads. Payloads are no longer associated with locations in the JSON and do not need to be defined in the schema.
  • Version 1.0.3 improves handle minting configuration to allow handles to redirect to the Cordra UI, the JSON of the Cordra object, payloads of the Cordra object, or URLs included in the JSON. There is also a handle updater to allow changes to handle records to be performed in bulk.
  • Version 1.0.2 includes a bug fix that prevented groups from referencing users correctly.