Configurable CQL rules options - Complex Event Processing

Hi,
I have created a CEP application which has CQL rules which checks the event attribute value against a constant value e.g 20
I have come up with 2 options to handle this scenario -
1) using hardcoded value of 20 in the CQL rule
2) using a table source node attached to the processor to retrieve this constant value from the table
In case of both the above options if the constant value changes then we can change the value directly in CQL rule using CEP visualizer (in case of option 1) or change value in the relational table (in case of option 2)
But in case i have many CQL rules(around 50) which uses the same contant it would not be feasible to change the constant value in all the rules using CEP visualizer which is a manual task.Instead we should have a option of managing the constant value at one place as part of CEP application and any change in value will reflect in all the rules.
Also in case of option 2 if we have huge volume of data coming in and at each occurence of event we will be retrieving constant value from table which might degrade the performance of the application.
Please provide any thoughts/best option here on how to deal with the configurable values in CEP considering the above scenarios.
I have also heard of using cache loader mechanism(using file or database) to load any configurable data in cache and use it in the CQL rules.but in case of any changes to the values we might have to re-deploy the application so that latest values are updated in the cache to be used in CQL rules. Is this correct ?
I appreciate any inputs here as i am stuck and unable to proceed until i finalize the approach.
Thanks 

Hi,
CQL doesn't support parameterized queries in the 11.1 release, so I would recommend one of the following:
1) store the value in a local cache and use a join to retrieve the value in your CQL queries.
2) use a user-defined function to retrieve the value in your queries.
3) insert the value into a CQL view and use a join to retrieve the value in your CQL queries.
If you store the value in a cache, access will be fast because the lookup is in-memory and in-process. If you change the value in the cache at runtime, the new value will be pulled from the cache immediately by the queries going to join, so you don't need to redeploy the application.
Hope that helps.
Seth 

Hi Seth,
Thanks for your response.Since i am new and still exploring CEP, I have few follow up queries on the options you have provided.
How the value in cache can be changed in the runtime ?
I am assuming that local cache can be loaded from file or database using cache loader.The file/database has name value pairs which is loaded in cache as key-value pair.The value from the cache can be accessed using the key-name.Once the application is deployed the initial data in the file or database will be loaded in cache.
Later how any change in the value in file/database for any name is reflected in cache in runtime ?
I have also read that in case any name/object not available in the cache it calls the load() method to load data in cache from file/database.
Is there any configuration required to achieve this ?
Also regarding the option to use table source,is it the correct justification that it will downgrade the application performance in case of number of events coming in and should not be used here ?
I appreciate your view on this.
Thanks
Sandeep 

How the value in cache can be changed in the runtime ?There are a number of ways to do this and the best way may depend on your application. In general, I would recommend that you write an MBean that exposes a method that allows a new value to be set. The implementation of the MBean would update the cache. You should be able to use Spring and make the MBean a spring component in your application bundle. The configuration system for the server is also extensible, so you could add some custom configuration for your spring component and configure an initial value in the same configuration file where you configure other things, like rules, etc. Unfortunately, the steps required to extend the application's configuration are not well documented, but feel free to ask questions if you decide to go this route. I can send you an example to get started. You could also store the initial value in a database if you want, or even configure it as a property directly in Spring.
is it the correct justification that it will downgrade the application performance in case of number of events coming in and should not be used here ?Yes, I believe you will see decreased performance. I'm not aware that we cache the result of looking up the value in the database. 

Appreciate your reply. I have created a Mbean as a spring component in my CEP application bundle and registered it. I have also created a local cache in the same CEP application.
I know we can use cache-loader to load values in cache while application start up.But is there any way to provide these initial values in the spring-context file ?
Now the other question is to update the cache values from the Mbean method which is exposed.Please share any sample code to access the local cache and update the values based on the input provided to Mbean method ?
Since CEP creates an Mbean for each component in CEP application (cache,processor) , can we update cache values using this MBean.
Please share any thoughts or details to resolve the issue.
Thanks
Edited by: user11166112 on Dec 31, 2009 12:00 AM

Related

Standard way to handle Repository in JPA

Hi All,
Consider the following scenario.
There is a table in db whose content together form as a repository. This table will be updated (update of existing entities, addition of new entities or removal of entities) rarely. So currently the approach that I was using was to create a singleton XXXRepository pojo which will read all the rows (entities) from XXX table and store them in a map.
If someone is updating the XXX table then after the update part of code is executed, a post runnable will run which will clear the repository (map) and hence next time any entity is being located from the map, the loadRepository will be invoked (as the map will be null).
This the way I currently handle repository caching which I can use through out the application life cycle.
Is there any standard way provided/supported in JPAto achieve/implement this requirement.
I went through the caching mechanism (both 1st and 2nd level cache). But it seems to be for entities or queries or data and for different purpose/approach than what is expected to be happening or achieved by Repository. Also if we are able to achieve this somehow by using 2nd level cache, then there is one more issue that cache updates only in case of jpa operations and jpql queries. In case of jdbc or native queries it will fail to be updated and we will have stale data (correct me if I'm wrong).
Please help me in this regards as what is the standard way to be followed in jpa for this.

Coherence Usage

I am comimg up with a problem related to coherence handling and usage .
My design is that i want to make a cache that would be having a key but would map to multiple values,
such that with a single key i would be able to get multiple different records
Real use case is as follows :
1.
A backend service is providing me an different session_id for a similar request every time i make a call to that session service.
2.
To fully utilize the session_id time usage window, i want that, every time a backend call is made, session_id's generated would be stored up in the cache,
with a same key "ACTIVE" in a cache and after the workflow gets over, i again wanted that session_id's to be updated in the cache but with a different key "IDLE"
3.
Would it be possible to have '1' to 'Many' mapping possible in cache, here 1 refers to cache key ("ACTIVE/IDLE") and many refers to cache values(session_id's)
Cache_1 FIRST CASE:
Key Value'S
ACTIVE 1234
ACTIVE 9876
ACTIVE 8889
ACTIVE 5555
ACTIVE 7787
AFTER THE FLOW IS OVER AND WE NEEED TO UPDATE THE CACHE
Cache_1 SECOND CASE :
Key Value'S
IDLE 1234
IDLE 9876
IDLE 8889
I am not using the inbuilt coherence support provided by OSB but i am writting my own java api(tangsol) to implement cache and used those classes in osb using java callout, till time i am able to implement one to one mapping for cache ,that is different key for storing different value in cache , my question is would it be possible to have one to many mapping possible in cache (one here refers to key and many refers to cache values )
Please provide suggestion to handle the above case
Regards,
Abhinav Gupta 
I am sure you know that you can store in Coherence any kind of object, as long as it is POF serializable... this means that to a single key you can associate a collection of values ... so probably this is not actually what you aim to achieve, uh? 
Hi Pierluigi Vernetto,
What i am trying to that is to reterive multiple records with a single key from cache , i dn't know how to write cache configuration to handle this case , if u have any sample please share to solve 1 to many mapping
Regards,
Abhinav Gupta

Result Caching in OSB

We are planning to implement OSB result caching feature in our project.We did the following to do a POC.
1.Created a DBAdapter to select from a table and created a BS out of that.
2.Enabled Result caching with TTL for 5 mins.
3.Invoked the BS from a PS.
4.Tested the PS by invoking from test cosnole.
5.Response was received as expected.
6.Changed the value in the table and tested again within 5 mins.
7.New values were returned instead of the ones in the cache.
What might be the problem?Should it not return the old value from table? 
The way OSB result caching works is based on your cache configuration of the cache token expression.
Now if the business service locates cache result through a cache key, it returns this result to the client instead of invoking an external service.
But if the data for the cache key has changed or the cache has has expired (depending on the configuration of expiration time) , the business service invokes the external service, returns the result to the client, and stores the result in cache.
As you have changed the data, I think the cache key has also changed and hence it is retrieving the new data instead of the old one.
Thanks,
Patrick 
Thanks Patrick.
//The way OSB result caching works is based on your cache configuration of the cache token expression.
I left the cache token exp blank
My understanding was that the same result would be returned even if the data has changed until the time of cache expiration set for the business service.
Was my understanding wrong? 
Each cached result is uniquely identified by a cache key that is made up of the ServiceRef (the unique identifier for the service which is the fully qualified path name of the service), the operation being invoked, and a cache token String. The cache token helps to uniquely identify a single cache result among other cache results for one business service. You control the value of the cache token. You can set the cache token either by configuring the Cache Token expression in the Result Caching configuration for the business service or by using the cache-token metadata element in the $transportMetaData using the proxy service message flow. If there is no cache-token defined then caching may not work as expected.
Please refer -
35.7.5 Improving Performance by Caching Business Service Results at http://download.oracle.com/docs/cd/E17904_01/doc.1111/e15867/configuringandusingservices.htm#CHDDCGEE
http://blogs.oracle.com/MarkSmith/2011/03/osb_and_coherence_integration_1.html
Regards,
Anuj 
Thanks Anuj.
It works now.I have given an expression from the request variable as a key.
Are there any specific WLS settings or JVM settings to be done to make caching work efficiently? 
By selecting the appropriate TTL you may have better performance. No specific settings may be required at JVM level.
Regards,
Anuj

How to refresh JPA cache from non-JPA code(JDBC)

In the system, let say there are two different tools are available, Tool-A_Uses_JPA and Tool_B_Uses_JDBC.
As the name of tools is self explanatory what persistence technology used by tools.
Say,there is a table COMMON_TABLE. This can be used by both tools to perform CRUD operation.
Lets see the problematic scenario-
Tool_B_Uses_JDBC tries to insert some new rows in COMMON_TABLE, by performing following tasks--
(Tool_B_Uses_JDBC already calculated how many rows will be inserted into COMMON_TABLE. Let's say it is 5.)
1. Read JPA_GENERATED_KEYS table for getting last_value for COMMON_TABLE. Let's say it is 30.
2. Add (last_value + number of rows to be inserted), here it 30+5 =35.
3. Update the JPA_GENERATED_KEYS table for last_value of COMMON_TABLE by the value calculated in step 2.
In my example it was updated by 35.
Above three steps done in a single transaction.
Now, ideally if any other tool use this COMMON_TABLE to perform any insert operation, the last_value from
JPA_GENERATED_KEYS should be 35. But for Tool-A_Uses_JPA, JPA cache is enabled. So this tool still get
last_value as 30, rather 35. And hence Tool-A_Uses_JPA, does not work properly.
So to resolve this issue, i thought to write a code to update JPA cache while updating the last_value.
Is there any way to update and/or send notification to all other tools to update JPA cache when updating
last_value form NON_JPA code?
Or is there any other way to resolve this issues?
Thanks. 
Marking this as answered, as this can never be a situation the way JPA uses caching. This was issue with the transaction form non-JPA code. Actually when JDBC uses data source then whether transaction will be committed as soon as application issues commit, is not guaranteed. Hence need to work on transaction.

How to pre-load Coherence Caches used within an OEP Application

Hi OEP/Coherence guys, I'm currently developing an OEP application that was consuming database inputs in CQL queries.I've replaced database direct access by Coherence caches access. My Coherence Local caches use a cache loader to fetch rows (by key) when there is a cache miss. This is working well, and the caches get filled in during the execution of my OEP application.The problem is that if CQL queries are made on some attributes (not the key) of not-yet-cached data, the load method of my cache loader is not invoked and there is no result to my CQL query.  I'm wondering how to pre-load my data in Coherence Caches, from the database, when the OEP application starts to avoid such kind of problems...Thx for any advice.Renato 
Check out the "Hot Cache" capability of Coherence which integrates with Golden Gate to get changes from the database OR set-up a "cache-loader" to load data into your cache when the OEP application starts. 
Hi. Could you please describe the way to "set-up a cache-loader to load data into your cache when the OEP application starts" ? I have a cache-loader configured with my cache. My cache-loader implements the "com.tangosol.net.cache.CacheLoader" interface.This interface only defines 2 methods:load(java.lang.Object oKey) ==> Return the value associated with the specified key, or null if the key does not have an associated value in the underlying store.loadAll(java.util.Collection colKeys) ==> Return the values associated with each the specified keys in the passed collection.None of these methods allows me to pre-load my data (and BTW it looks like "loadAll" is never called by OEP) Thx RP

Categories

Resources