MuleSoft Object Store: Tutorial, Examples & Best Practices
In today's digital world, businesses need fast, reliable, and smooth data flow among applications, APIs, and services to succeed. MuleSoft, a top integration tool, makes this possible with its Object Store feature, which helps developers efficiently store, retrieve, and update temporary or long-term data within a Mule application.
Object Store is a key-value, pair-based, in-memory, or persistent storage mechanism that enables Mule applications to manage internal state, cache it, or share it across Mule applications. It enables stateful patterns like OAuth token lifecycle management in stateless API design and architectures. Unlike databases, it is optimized for high-speed reading and writing, which makes Object Store an easy choice for scenarios where low-latency data access is essential.
This guide delves into the mechanics of Object Store to explain what it is, how it works, and why you may want to use it, from basic configurations to enterprise-grade implementations.
Summary of Mulesoft Object Store concepts
What is an Object Store?
Mulesoft Object Store is a simple yet fast and network-efficient storage system that uses key-value pairs. It allows Mule applications to store and access data from any application part whenever needed. REST web services are usually stateless; they don’t store any data from previous executions. An object store can retain data beyond the lifecycle of a Mule flow, so it can be used to implement a stateful web service in a general stateless architecture of MuleSoft.
Object Store supports time to live (TTL), a valuable feature that automatically eliminates unnecessary data. It is also scalable.
{{banner-large-graph="/banners"}}
Why use Object Store in MuleSoft?
MuleSoft Object Store is a powerful feature that temporarily or permanently stores data that an application frequently needs during execution. Improving the performance, reliability, and efficiency of integration workflows is vital.
For example, this data can be stored and retrieved quickly when needed instead of making repeated API calls to fetch the same data, like an OAuth access token or user-session information. This reduces unnecessary network traffic and speeds up response times.
Additionally, when an object store is set to be persistent, the data is saved on disk and survives even if the application restarts, which is essential for critical workflows. For instance, if a JWT token is stored in a persistent object store and the app crashes, the token will still be available after the app returns online, preventing dependent API calls from failing. Such tokens would be lost without the Object Store, and the workflow could break.
In short, Object Store helps in caching, state management, and reliability, making your MuleSoft applications more robust and efficient.
Use case: Storing and reusing an OAuth token
When integrating with an external API (like Salesforce, Google, or LinkedIn), you often need an OAuth access token to authenticate each request. These tokens are usually valid for a limited time (like 1 hour). Fetching the token every time by calling the authentication server leads to unnecessary network traffic and slows down your application.
Suppose you have a MuleSoft API that fetches customer data from Salesforce. To do that, an OAuth token is needed. Instead of requesting a new token from Salesforce on every call (which consumes time and resources), you can do the following:
- Check Object Store to see if a valid token already exists.
- If the token is found and valid, use it to call Salesforce.
- If it is not found or it has expired, request a new token from Salesforce, store it in the Object Store, and use it for both current and future calls.
Here’s the MuleSoft flow:
[HTTP Listener] → [Check Object Store for Token] →
[If Token Exists] → [Use it to Call Salesforce]
[Else] → [Fetch New Token] → [Store in Object Store] → [Call Salesforce]
How to use Object Store in a MuleSoft application
In MuleSoft, you can use ObjectStore by utilizing the ObjectStore module provided by MuleSoft as shown below.
First, you need to add the Object Store dependency by clicking the Search in Exchange button from the Mule Palette:
If the installation is successful, then we should be able to see all Object Store operations in the Mule Palette like this:
Basic operations and code example
As you can see in the Mule Palette, the object store offers the following basic operations in the form of connectors in the Mule palette:
- Clear: Removes all the contents in the store.
<os:clear objectStore="myObjectStore"/>
- Contains: Checks whether data exists in the object store, returning true/false.
<os:contains
key="user_abc"
objectStore="myObjectStore"
targetVar="isKeyExists"
/>
<choice>
<when expression="#[vars.isKeyExists]">
<logger level="INFO" message="Given key exists!" />
</when>
<otherwise>
<logger level="INFO" message="Given key not found." />
</otherwise>
</choice>
- Remove: Removes the key-value pair from the object store.
<os:remove key="user_abc" objectStore="myObjectStore" />
- Retrieve: Retrieve data from the object stored by key.
<os:retrieve key="user_abc" objectStore="myObjectStore" targetVar="retrivedUser" />
<logger level="INFO" message="Retrived User: #[vars.retrivedUser]" />
Note: If the key doesn’t exist, an `OS_KEY_NOT_FOUND` exception will be thrown
- Retrieve all: Fetch all stored data.
<os:retrieve-all objectStore="myObjectStore" targetVar="allRetrivedUsers" />
<logger level="INFO" message="All retrived User: #[vars.allRetrivedUsers]" />
- Retrieve all keys: Fetch all available keys.
<os:retrieve-all-keys objectStore="myObjectStore" targetVar="allKeys" />
<logger level="INFO" message="All keys: #[vars.allKeys]" />
- Store: Store the given value against a particular key.
<os:store objectStore="myObjectStore" key="user_abc" targetVar="allKeys">
<os:value ><![CDATA[#[vars.user]]]></os:value>
</os:store>
Advanced operations and code example
Let’s look at an example of locking an object store to avoid race conditions during parallel writes. MuleSoft applications running in multi-threaded or multi-worker environments risk parallel object store writing issues when multiple threads or workers attempt to access or update the same shared object store simultaneously. These issues can lead to data corruption, inconsistencies, or unexpected behavior, especially in distributed systems where Mule applications are deployed across multiple workers in a cluster.
As there is no out-of-the-box solution for locking operations in a parallel writing scenario, MuleSoft recommends using distributed locking via LockFactory.createLock(). Example code using Java could be like this:
public void incObjectStoreValue(String objectStoreName, String objectStoreKey) throws ObjectStoreException {
Lock lock = muleContext.getLockFactory().createLock(objectStoreName + "Lock");
ObjectStore<Integer> objectStore = muleContext.getObjectStoreManager()
.getObjectStore(objectStoreName);
lock.lock();
try {
Integer val = 0;
if (objectStore.contains(objectStoreKey)) {
val = objectStore.retrieve(objectStoreKey);
objectStore.remove(objectStoreKey);
}
objectStore.store(objectStoreKey, val + 1);
} finally {
lock.unlock();
}
}
A small Groovy script can be used here to do the object store write operation with an appropriate lock:
<scripting:execute engine="Groovy">
<scripting:code ><![CDATA[registry.lookupByName("beanName").get().incObjectStoreValue(vars.osName, vars.osKey)]]></scripting:code>
</scripting:execute>
Set an appropriate TTL value to avoid stale data and memory overflow, leak, or bloating. The TTL can be either rolling or static. If a particular value is specified for the TTL, it is considered static; otherwise, Mule versions 4.2.1 and later set a rolling TTL by default. For example, accessing the data during the last seven days of a 30-day window extends the TTL for another 30 days.
Object Store v1 vs. Object Store v2
There are two major versions of Object Store, which have significant feature differences. Object Store v1 provides basic key-value storage but lacks auto-expiration, proper clustering support, and limited storage capacity. Object Store v2 offers substantial improvements over v1.
{{banner-large="/banners"}}
Different types of object stores in Mulesoft
Persistent object store
Persistent storage is the default type of object store. It's just a checkbox in the store configuration.
A persistent object stores persistent data on the disk that can be retrieved even if the system goes down. It's a long-term storage type that ensures data recovery after a restart.
On the downside, as disk writes have some overhead, a persistent object store is unsuitable for high-transaction systems. Disk access can be a bottleneck for achieving large values of transactions per minute.
There are a few parameters that can be set here:
- Max entries: This value sets a size boundary that indicates the maximum number of entries allowed in the object store. If the number exceeds this number, the extra entries will be removed in the next expiration thread run.
- Entry TTL: This number indicates a static TTL value, after which the data will be evicted from the store. Otherwise, a default value will be set depending on the object store version.
- Expiration interval: This number indicates how frequently the expiration thread will run. The default value is 1 minute. These parameters depend on the previous two; if neither is set, then putting an expiration interval has no effect, and the thread will not run.
In-memory object store
The opposite of persistent data is transient data, which can be achieved by deselecting the “Persistent” check box. That data will now be available until the system goes down. This type of store is suitable for high-performance computing where short-lived and easily reproducible data, like an authorization token, needs to be stored.
Custom object stores
MuleSoft supports custom object stores, user-defined storage that extends or replaces the object store's default capabilities. This feature allows several customizations depending on the user's needs, like alternative storage backends (Redis, DynamoDB, RDBMS with transaction, etc.), custom TTL logic, and data encryption and decryption.
As a customization example, an object store with Redis is explained here.
First, add the Redis dependency, as shown below.
Next, configure the object store with Redis as the global configuration below.
Now, the object store will read and write from the Redis located in `redis.host`.
Real-world scenarios using Object Store
Setting up watermarks
Watermarking is a powerful technique for efficient data processing in MuleSoft. It marks the state of a data source at a specific time to track the progress of a periodic synchronous job. This method prevents the processing of already processed data and helps focus only on new or updated information. An object store is commonly used here to store the data of the last run so that the synchronous process can start from exactly where it ended the next time.
Here are the steps involved in watermarking:
- Initialization: When data processing just starts, an initial watermark needs to be set. The most commonly used watermark is a timestamp, but an identifier or any other information representing the current data state can be used here.
- Data polling: Data polling from the data source is compared with the existing watermark to ensure that only new or updated data is processed.
- Data processing: Only new or updated data is processed to ensure efficiency.
- Update: Finally, the watermark needs to be updated so that the process can work with data after the current process is processed next time.
The following images and code illustrate these steps.
<os:object-store name="watermarkStore" persistent="true"/>
<flow name="userDataSync">
<os:retrieve key="watermark" objectStore="watermarkStore" target="watermark">
<os:default-value>1970-01-01T00:00:00.000Z</os:default-value> <!--Init with Epoch time / beginning of the time -->
</os:retrieve>
<db:select>
<db:sql ><![CDATA[Select userId, userName, address, email, phone, lastUpdateTime From UserTable where lastUpdateTime > :watermark]]></db:sql>
<db:input-parameters ><![CDATA[#[{
"watermark": vars.watermark
}]]]></db:input-parameters>
</db:select>
<flow-ref name="processTheData" />
<os:store key="watermark" failIfPresent="false"
failOnNullValue="false" objectStore="watermarkStore">
<os:value>#[max(payload map $.lastUpdateTime)]</os:value>
</os:store>
</flow>
This dataSync flow can be periodically triggered by a <scheduler/> component to make an efficient ETL job.
Stateful OAuth token lifecycle management patterns in stateless API design using caching
In a stateless API architecture, every request is independent of the others, and no storage is maintained on the server side. However, OAuth token management requires maintaining token states like token expiration, revocation, and refreshing. Mulesoft Object Store can be a centralized, fast cache for OAuth tokens while the API architecture is stateless.
Implementation steps:
- Token creation and caching: When authenticated using the/oauth/token URL, the authentication API generates an access token and a refresh token. These tokens are stored against the clientId, setting a proper TTL that matches the expires_in value of the API response.
- Token validation: Before every API request with a bearer token, verifying whether the token exists in the object store is necessary. If the token expires, a token refresh is needed; if it is manually revoked, use an access denied error.
- Token refresh: Before token expiry, the client triggers a refresh request using the refresh_token.
- Token deletion at logout: When the user logs out, you must delete (revoke) the token from the object store.
CurieTech AI has similar capabilities but is more advanced. It can detect and debug errors, find anti-patterns in code, and offer intelligent fixes. CurieTech AI can generate efficient object store caching solutions without building from scratch, saving time.
To start, describe the task straightforwardly: “Create an object store to implement OAuth cache.”
Then, export the output and import it into Anypoint Studio.
The code generator creates both the get and refresh token flow as shown below:
Object Store limitations
Mulesoft Object Store is designed for lightweight key-value-based storage, primarily for temporary purposes or caching. It has a few limitations that need to be considered before using it:
- An object store is unsuitable for a large dataset requiring complex queries. As data grows, the performance of the object store degrades because it lacks indexing features. Unlike databases, data joining and aggregation—like `group by` and `order by`—are impossible here.
- Object Store is not a good choice when transactions, rollback, or data atomicity are needed. Data inconsistency is risky because it lacks these ACID (atomicity, consistency, isolation, and durability) features.
- Object Store does not replace a database or an extensive caching system. Alternate suggestions include databases like PostgreSQL or MySQL or distributed caches like Redis or Memcached.
Object Store Usage in Anypoint Access Management Console
In the Anypoint access management console, you can monitor the object store usage to track the API request consumption against your subscription limit. The usage is shown in a chart indicating how many effective API requests your apps have consumed and how many effective API requests your subscription allows. The maximum number of permitted requests depends on your subscription type:
Object Store v2 continues to work if the usage exceeds the limit, and Mulesoft notifies the account administrator. Here's how to access and interpret the Object Store usage information:
- Click the Object store in Access Management under the Subscription section
- The bar chart displays your practical API request usage for the current and previous months.
- In the circle chart, the usage percentage is shown. If your usage is below 1%, it will display "<1%". The circle graph will turn red as your usage approaches your quota.
- This page also shows the subscription type (Base / Premium).
Object Store best practices
Mulesoft Object Store is a powerful tool, but its improper use could cause performance issues, data inconsistency, or memory overflow. Here are some best practices to follow to ensure optimal results:
- Use an in-memory object store for high-throughput applications because persistent mode has some writing overhead.
- Optimize storage by setting an appropriate TTL (time-to-live) to avoid memory bloat.
<os:store key="#[vars.cacheKey]" value="#[payload]" ttl="3600000" />- Don’t store everything in one place. Data partitioning is a good technique for reducing the data search space and optimizing data access. For example, you can use `token_store` to store the token and session info in session_store.
- Don’t store large files; instead, store a file reference.
- Remember that the Object Store’s capacity is limited and not designed for a large volume of data.
- Object Store can cause problems with concurrent updates and data discrepancies in a multi-worker environment without a distributed lock.
- Beware of rate limits for transactions.
- Object Store is not secure. Use secure properties for sensitive data.
Automatic code generation using CurieTech AI, tailored to your requirements, ensures that your scripts are efficient and follow best practices. The screenshot below depicts the CurieTech AI, which auto-creates an object store that follows a set of requirements provided in simple English. The task is described here as “Implement watermarks using an object store to perform the DB migration ETL task.”
{{banner-large-table="/banners"}}
Conclusion
Mulesoft Object Store is a flexible yet powerful caching and temporary persistent mechanism. If developed following best practices, it can give an organization an efficient, high-performance, scalable, and resilient integration solution. Using in-memory capability, proper TTL-based auto expiration, and distributed caching using Redis, developers can optimize API performance, manage the token lifecycle, and develop stateful workflows in stateless architectures while avoiding common pitfalls.