Examples of programatic creation of EPN elements - Complex Event Processing

Hello,
I'm looking for examples of how to create things like Channels, CQL Statements, etc programatically, not just statically creating them using the EPN editor in eclipse.
For example:
I want to have a static processing network that just takes all events and passes them to a bean. The bean then looks at the events and decides that a new channel should be created for something in this event (say event type, or subject). So it would dynamically create a new channel and pass that event on to it. Then any time it started seeing similar events, it would pass those on to this new channel as well. I'm not sure if this is a good idea or a good use of CEP or not though at the same time...
Here is another example:
We want to expose an API to allow clients to create new queries into the stream on demand. So we have predefined queries set up that handle default events. A new client comes in and wants to start getting certain events (content based routing basically). So they make a call to the API and say I want to start getting events with x in them. A new CQL query is created (select * from stream [now] where thing = x) and probably also a new channel/bean is created as a destination.
I can't find anywhere that describes how you do any of these things. You can do all this using Esper, so I assume Oracles CEP has a way to do it too...If you can help, that would be great. 

serff wrote:
I want to have a static processing network that just takes all events and passes them to a bean. The bean then looks at the events and decides that a new channel should be created for something in this event (say event type, or subject). So it would dynamically create a new channel and pass that event on to it. Then any time it started seeing similar events, it would pass those on to this new channel as well. I'm not sure if this is a good idea or a good use of CEP or not though at the same time... We don't currently expose any programmatic API for creating EPN elements - part of this is because the connection between elements is augmented in many different ways when the EPN is constructed and also because we optimize certain cases. What is your new channel going to do in this instance? There is nothing stopping you writing an event bean that manages its own "channels" (written by you), but it would help to know a few more details in order to make any recommendation here.
Here is another example:
We want to expose an API to allow clients to create new queries into the stream on demand. So we have predefined queries set up that handle default events. A new client comes in and wants to start getting certain events (content based routing basically). So they make a call to the API and say I want to start getting events with x in them. A new CQL query is created (select * from stream [now] where thing = x) and probably also a new channel/bean is created as a destination. The JMX API allows you to create queries dynamically - this is what wlevs.admin does. This may be the easiest way to manage dynamic queries. You could also do this directly by making a bean ApplicationContextAware (a Spring class) and then from that get the BeanFactopry and then the bean you are interested in, cast to the appropriate API and add rules. Unfortunately the java API for the processor is not public, so you would be on your own trying this approach. 

Hi,
As suggested, the simplest option is to have an event-bean use com.bea.wlevs.management.configuration.CQLProcessorMBean to dynamically manage the queries (or EPLProcessorMBean if you are using EPL).
If you must dynamically create a new 'EPN', one suggestion is to dynamically deploy a new application that receives the downstream events from the original one. Again, JMX can be used for this purpose.
Cheers,

Related

WIPEDIT Request types/ EWPS modifyFolder

Hi,
What is the difference between the following two iDocumaker request types and how/where are they used?
[ReqType:i_DPRModifyWIPData]
[ReqType:i_ModifyWIPData]
Here is the situation:
We are using EWPS web services along with WIPEDIT Plugin in our implementation. The requirement is for application to control the forms set selected during the policy issuance process so that they can:
1. Add/remove the forms in the WIP (through EWPS call)
2. Refresh the input data supplied from the application such that User entered data need not be re-entered. Only the Extract (mapped) data should be refreshed.
We are using the modifyFolder EWPS webservice with an updated composeData for this. However, this is not invoking all the Rules (gendata) to run again such that appropriate pagination is maintained along with any other custom form level dal scripts.
I am thinking if we can invoked either RPDRun or DPRModify, it should work. But I don't know how to do that.
Looking desperately for some suggestions.
Thanks,
Prashant 
I ran into something similar with an exception. I was trying to invoke RP again after WIP editing during a complete. I ended up creating a custom request type that i call via doCallAPI. I basically created a request that gets the WIP entry as xml and runs RP with another afgjob file. This allowed me to process data / pagination etc after the initial call to RP that created the WIP record in the first place. the bridge book and the rules book are good places to start. another thing to consider is running this extra processing in your own application and using the doModify to produce the end state of the transaction. this would entail manually adding forms and images as well as manually setting field values etc and sending the final compose data object in the doModifyFolder request.
[Documaker Documentation Library|http://www.oracle.com/technetwork/documentation/insurance-097481.html]
All of the documentation for many versions can be found there. 
Thanks for the response. This is extremely helpful.
Can you please send the rules included in your custom request called via doCallAPI? I think I should be able to replicate what you have done and get over the issue.
We already have a custom Java routine to update the ComposeData to be used during modify.
Looking forward to hearing back 
What is the difference between the following two iDocumaker request types and how/where are they used?
[ReqType:i_DPRModifyWIPData]
[ReqType:i_ModifyWIPData]The base system request type is i_ModifyWIPData. I think the+i_DPRModifyWIPData+ is a copy or modified request type. Your base request type should look like this:
[ReqType:i_ModifyWIPData]
function = atcw32->ATCLogTransaction
function = atcw32->ATCLoadAttachment
function = atcw32->ATCUnloadAttachment
function = dprw32->DPRSetConfig
function = dprw32->DPRInitLby
function = dprw32->DPRGetWipFormset
function = dprw32->DPRLoadXMLAttachment
function = dprW32->DPRUpdateFormsetFromXML
function = dprW32->DPRModifyWipData
Here is the situation:
We are using EWPS web services along with WIPEDIT Plugin in our implementation. The requirement is for application to control the forms set selected during the policy issuance process so that they can:
1. Add/remove the forms in the WIP (through EWPS call)
2. Refresh the input data supplied from the application such that User entered data need not be re-entered. Only the Extract (mapped) data should be refreshed.
We are using the modifyFolder EWPS webservice with an updated composeData for this. However, this is not invoking all the Rules (gendata) to run again such that appropriate pagination is maintained along with any other custom form level dal scripts.
I am thinking if we can invoked either RPDRun or DPRModify, it should work. But I don't know how to do that.One way to do that would be:
1) Create request type configuration for RPDRunRP which is configured to expect the formset XML as an extract file.
2) Create mapping of variables on forms so that user-entered data is preserved
3) Create triggers on forms so that forms in the transaction are triggered. (You can of course have multiple triggers, one for the policy issuance system extract data and one for EWPS ComposeData.
4) Create a configuration (AFGJOB, Extract layout, etc) that uses the ComposeData as input). This configuration should be set to ALWAYS kick transactions to WIP.
5) Your application should dump the formset XML (you could use ComposeData for this part) after changes to formset have been made.
6) Your application should invoke the doCallApi EWPS method, which should invoke the RPD request type (the base RPD request type should work) and specifying the CONFIG you created. Documaker will run using the ComposeData input, trigger forms and map data. The transaction will be kicked to WIP.
7) Your application should invoke the doFindFolder EWPS method using the KEY1/KEY2/KEYID information from the ComposeData request (this step may be optional depending on the version of Documaker you're using - some versions return the WIP ID when a WIP transaction is created).
8) Your application can delete the previous WIP transaction.
9) You can invoke WIPEDIT with the new WIP transaction and continue work.
This is a solution we've implemented before so it is known to work.
--Andy                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
Thanks Andy,
This is an excellent option and we have considered this option, but we ran into a different issue with this. Documaker has limitation on the length of the data field. It can only take upto 1000 characters of length. We have many user editable fields that are multi-line texts to be free-form texts for users. The user could type in the entire page during the interactive session. There is no way we could maintain the paragraph formatting as well with free-form texts and we have left it to the users to be responsible for that.
If there are ways to get past this limitation we can definitely make it to work.
Your thoughts..
Thanks,
Prashant 
we use i_pluginsave to have all of the entered user data saved to the wip form set. We have similar forms where the users select a bunch of paragraphs and also enter a LOT of free form text. we use i_pluginsave to have all of this info saved to the formset. Then when we do our complete we use i_getwipentry to get the form set as XML. we take the xml and send that to a separately defined RPD request as the extract file. the afg job file for this request takes the formset info and produces the print files (we do some stuff with recips and mailer pages etc). what we are doing is very similar to what Andy suggested with the exception that we do not kick the form set to WIP again. I do not have access to the system at the moment. I can supply them on monday evening if it would help.

non-editable category attributes

Hey there,
I would like to configure category attributes at the library/folder level to be non-editable. I can do this through the web interface by not selecting the editable check box.
However, I would like to (if possible) make them editable through the API, update their values, and then set them to non-editable. I'm calling the API from a bpel process - I basically only want the bpel process to update the attribute values, and make them read-only for other users.
I didn't see anything in the ApplyingCategories example, can someone offer any suggestions?
I'm amazed by the amount of code one must write to do simple operations through this API. The excessive use of nested arrays makes it very difficult to follow what is being done in the code.
Many thanks for any assistance
-Barry- 
Will the instances of the category class in question be created initially from the API (/bpel)?
If so, set all the attributes of the custom Category class when defining it to be non-updateable. This means, when an instance of the category class in question has been created, the initial values provides for the instance will be locked-in and read-only from that point onwards.
Or alternatively, set attribute overrides on the category attributes in the appropriate folders to make certain attributes non-updateable on a folder-by-folder basis.
If the user is populaing the initial category instance values, and your BPEL process is intending to merely update/validate these initial values, you could look to have the BPEL process create a new instance of the category in question, rather than update the existing user-populated instance.
Unfortunately there is not much else you can do - unless you want to change the user's access permission on the document, such that only specific administrators have the SetMetadata permission on the document, with all appropriate others just having GetMetadata.
This would require then that appropriate documents have a custom ACL which is quite expensive.
GetMetadata - get the metadata of an item (in other words, view category information) - CAPABILITY_GETMETADATA
SetMetadata set the metadata of an item (in other words, create, modify, or delete category information) - CAPABILITY_SETMETADATA
--------
re - amount of code required to do simple operations:-
The trade-off to Content DB's appraoch to utilizing rudimentary object types and patterns can be seen in Stellent's Web Services. Literally, with Stellent's services, every operation has its own response class which encapsulates more generic classes and so forth.
e.g.
PingServerResult psResult = manager.pingServer(props);
GetFilePlanResult fpResult = manager.getFilePlan(...);
GetFilePlanAllResult fpaResult = manager.getFilePlanAll(...);
CheckInExternalResult cieResult = manager.checkInExternal(...)
...
etcI 100% agree the CDB Web Services are very verbose - but one can create quite simply utility classes to remove some of the bulk and simplify processing. The other benefit you will find is that the API calls are identical whether called in-process, or via Web Services.
Matt. 
Thanks Matt.
Yes, the instances of the category class are being created from the bpel process.
In the process I call fileManager.updateDocument with a category definition. This is the first time a category instance is applied to the document. The process is triggered on an upload event so the createDocument is being handled/called behind the scenes I assume; and with no category instance applied to it.
It sounds like what I'm doing is similar to your third option, except the user is not applying a category instance to the document.
Because I've configured attributes of this category (at the folder level) to be non-editable, calling fileManager.updateDocument with a category definition has no effect, but making them editable does - however it makes them editable for other users which I don't want.
I sort of repeated myself with my first post, but wanted to clarify the point that the category first gets applied to the document on the updateDocument not the createDocument, so creating a new category instance and applying it wouldn't be applicable in this case...

Filter Value on a Channel

I have been trying to find this answer in an example or on a forum and I can't find it anywhere. I see a lot of people asking the question though and there is no answer. Anyway what I am trying to do is filter on a message channel with severla different apps and I can't come up with the filter value to use. I have an app that is sending events to an Event Generator. I can't build separate channels I guess since the differentiation for the events is in the message body but I think I should be able to filter. The message that comes to the channel is an XmlObject and it looks like:
<ord:orderTaskEventStatusChange xmlns:ord="http://www.xyz.com/MIP/OrderManagementEvents">
<ord:eventName>CONFIG</ord:eventName>
<ord:eventId>1000</ord:eventId>
<ord:documentNumber>25304</ord:documentNumber>
<ord:servItemId>-1</ord:servItemId>
<ord:taskNumber>141742</ord:taskNumber>
<ord:userId>app_api</ord:userId>
<ord:customerId>-1</ord:customerId>
<ord:taskStatus>W</ord:taskStatus>
<ord:eventVersion>79</ord:eventVersion>
<ord:retryAttempts>5</ord:retryAttempts>
</ord:orderTaskEventStatusChange>
What I want to do is place filters on the subscription of the apps to the channel such that I can filter only on specific events. I really don't want to build a distributor that sends events to other apps but I can't get the syntax right for a filter. Can anyone help? Please 
On which field(s) do you want to filter the messages? 
Hi.
When you create the subscription control in your process project you check the "This subscription will be filtered" button at the bottom of the dialog.
WLI will generate a subscription template for you. In this template you need to specify the field you want to filter upon. This is done in an Xquery:
/*
* #jc:mb-subscription-control xquery::
* declare namespace ns1="http://www.xyz.com/MIP/OrderManagementEvents"
* data($message/ns1:order/ns1:eventName)::
*/
You can also use the XQuery wizard found in the mb-subscription-control pane in the Propery Editor.
An alternative solution is to create a process that extracts the eventName and publishes the incoming file to the appropriate channel. Then each of your processes can subscribe to its "own" channel without any correlation.
Regards Maskefjes 
Thanks for the response. I think that would work but we also found another way. If you place this in the channel file:
<p><channels xmlns="http://www.bea.com/wli/broker/channelfile"
          channelPrefix="/MIP"
          xmlns:eg="http://www.bea.com/wli/eventGenerator"
          xmlns:dp="http://www.bea.com/wli/control/dynamicProperties"
          xmlns:oagpo="http://www.openapplications.org/003_process_po_007"
xmlns:ord="http://www.xxx.com/MIP/OrderManagementEvents">
<channel name ="outboundEventChannel" messageType="xml"
<p><b>qualifiedMessageType="ord:orderTaskEventStatusChange"/></b>
     
</channels><p>
and have that schema available in the project the schema for the orderTaskEventStatusChange will be in the filter box in the control when you place it in the work flow. Then it is just a point and click to add a filter and stick the value for the eventName you are looking for in the Filter Value field.

Multiple Tables/Update

Hi,
We have a situation, where records in 5 tables need an update. What is the best possible way to handle this?
1) Have one function # the logical level which handles the message transformation required for each update and sequentially call update method of all the 5 Physical Services
2) Expose 5 functions at the Service Bus level; and let Service Bus handle the Message Transformation for each update
The advantage with 1st is no back and forth calls with the client or intermediate service. Are there any -ve points for the 1st one.
Thanks,
Is 
To answer and your other question - Here is some reading material for you :
http://download.oracle.com/docs/cd/E13162_01/odsi/docs10gr3/odsi10gr3/How%20To%20Develop%20Good%20XQSEs.html#HowToDevelopGoodXQSEs-Employee.ds%2CPartTwo%3AHowToUseXQSEProcedures
I have best luck finding this with Google using searches like :
site:oracle.com data service integrator update xqse mapper
We have a situation, where records in 5 tables need an update. What is the best possible way to handle this? If you have a single dataservice that retrieves data from those five tables, the generated updateYourDataService() function should work - you should not have to write anything. If the generated updateYourDataService() function doesn't behave as you wish, or if ODSI is unable to generate it - you can use the Update Mapper to specify the mapping that you want.
Otherwise - you have already identified the pros/cons of (1) vs (2). (1) will perform better, and (2) will be simpler to implement.
- Mike

Associating dynamic query creation with a dynamic channel and event type

Hi,
I understand that queries can be added/modified dynamically using JMX API, without bringing the CEP application down. But because a channel (relation or stream) has only a 1:1 mapping with a event type, any new query addition needs creation of a new channel and event-type. How can this be done dynamically - basically, dynamic creation of a EPM component.
Thanks. 
Just to add to this - there is a related thread on this forum
Re: Examples of programatic creation of EPN elements
It has the following statement "There is nothing stopping you writing an event bean that manages its own "channels" (written by you)"
How can I write a event bean that manages channels - I am taking that manages here means "creating" channels. Please let me know 
No. You can't dynamically create new channels in the EPN, but you can dynamically assign the value of the "selector" of an existing channel via JMX. By doing this, you can control where the output of any new queries that you create will go (i.e. which existing channel). 
To pre-create a channel, shouldn't we know what kind of events the channel will contain? One does not know the event-type of the query output beforehand and it may not map into existing event-types. Because of 1:1 mapping between channel and event, we cannot create any anticipatory channels before hand.
Please confirm as this is a crucial requirement for us. 
Hi,
You can't create channels dynamically within an application, however you can dynamically deploy new applications that provide new channels to a processor of another application.
For example, say you have a processor called helloworldProcessor in an application named helloworld.
You can deploy a new application that has the following EPN:
<wlevs:channel id="newHelloworldOutputChannel" event-type="HelloWorldEvent" advertise="true">
<wlevs:listener>
<bean class="com.bea.wlevs.example.helloworld.HelloWorldBean"/>
</wlevs:listener>
<wlevs:source ref="helloworld:helloworldProcessor"/>
</wlevs:channel>
Hope this helps,
Alex 
Thanks for the reply. Even if we can dynamically spin off a new channel in a new application, your explanation is assuming that the event-type is still known beforehand. What if my dynamic query does NOT output HelloWorldEvent but outputs data that is not known before hand? We need to think in terms of some generic data - something like a map of name, value pairs. How does CEP handle this use case? From the information I collected so far, it sounds like this is not supported.
I guess we need channels that can support a concept of a generic event type. 
Hi,
Not necessarily, the new event-type can be registered by the new application as well.
Further, you can build your own generic event-type by using com.bea.wlevs.ede.api.EventBuilder.
Hope this helps,
Alex 
Hi,
Even if it is possible to launch a new applicaiton (OSGI bundle) for each new dynamic query, it may not be practical from a scalability and performance perspective. 
Hi,
It does not need to be for every query, but only for those queries that output a different structure, which in all likelihood, would also mean you need a different Java class handling the event downstream anyway.
But, perhaps if it is the case that you have a generic Java sink, then the option to go for is indeed that of the EventBuilder route (i.e. generic event-type).
Hope this helps,
Alex 
Can you please show me an example how you visualize configuring a generic event-type (using EventBuilder) to a new channel in the new application (for a new query with a new output)? The event builder API will basically return an event of type Object. (createEvent() in com.bea.wlevs.ede.api.EventBuilder )
Thanks

Categories

Resources