CQL limitation - Complex Event Processing

Hi,
I can't find straigh answer for my issue : It's not RANGE, ISTREAM, DSTREAM, SLIDE...
My data in stream ( 0.5 sec for each row)
QUERY
_____ __ 0 sec
11111
22222 __ 1 sec
11111
11111 __ 2 sec
22222
22222 __ 3 sec
22222
11111 __ 4 sec
My window size is 2 secs i want to use "Group by" on Query column, take MAX() from COUNT() and remove all queries which are included into this result.
In second 2 my output will be 11111, 3. and stream should be 22222.
In second 4 my output will be 22222, 4 and stream should be 11111.
Do I need to write my own stuff for that.. or will have to move it into Java Queue within bean? I've tried to write GROUP BY which works, but without any update of stream.
--
R. 

I am not certain that I understand the requirement correctly.
Here is my understanding - could you kindly confirm if I have got it right
1) You want to process every 2 seconds
2) Each time you process, you will consider the entire stream (from the beginning of time) except for those queries that you had processed in previous rounds.
2.1) You will pick the query with largest count for processing
From your example, at t=4, you will look at all queries from the beginning of time i.e. 11111 - (count=4) and 22222 - (count=4). However, at t=2 , you had processed 3 counts of 11111, so while determining the count at t=4, you will exclude those.
So, you will have 22222 - (count=4) and 11111 - (count=4-3=1). Hence, at t=4, you will select 22222 (count=4) for processing.
Did I understand this right ?
Anand
Edited by: user785377 on Oct 12, 2009 2:15 AM
Edited by: user785377 on Oct 12, 2009 2:15 AM 

Yes I want to work on all elements is stream.
You're close to my concept, the think is that I want to delete form stream "ALL" rows which are used to
generate output result. ( COUNT )
Second 2:
Before:
QUERY
_____ __ 0 sec
11111
22222 __ 1 sec
11111
11111 __ 2 sec
Outpuit:
11111 COUNT=3
After:
QUERY
_____ __ 0 sec
.
22222 __ 1 sec
.
._____ __ 2 sec
Second 4:
Before :
QUERY
_____ __ 0 sec
.
22222 __ 1 sec
.
._____ __ 2 sec
22222
22222 __ 3 sec
22222
11111 __ 4 sec
Output:
22222 COUNT=4
After :
QUERY
_____ __ 0 sec
.
._______ 1 sec
.
._____ __ 2 sec
.
._____ __ 3 sec
.
11111 __ 4 sec
In SQL it's that type of operation :
MySQL version:
SELECT Query, COUNT(Query) AS ctr FROM Query GROUP BY Query ORDER BY ctr DESC LIMIT 1;
// Oracle ROWNUM, which will have to work on dynamic temporary table ( SELECT ... (SELEC ) rownum doen't work with aggregation sometimes.
DELETE FROM Query WHERE Query = <RETURNED Query>
Thanks
R 

The requirement here seems to be some form of custom aging from the Stream (aging out elements in an order different from the stream-order - which is the order in which the elements (queries in this case) were input to the stream).
Further, what needs to be aged out is determined by output of processing at an earlier point in time.
Here is an approach for this use-case
1) Model a CQL relation (instead of a stream) to hold exactly those input queries that are required for the next round of processing. Modeling as a relation enables updates / deletes to be performed. In particular, look at the queryEventRelationChannel in the detailed code included with this post.
2) Use a "TimerStream" (with input sent to it by a "TimerAdapter" class) to trigger processing as per your requirement (every 2 seconds)
3) Use CQL to maintain a running count of the queries and also maintain the queries in sorted (desc) order by count. This helps get the TOP query by count.
The derived relation (view - which evaluates to a relation) named R2 in the included code holds, at any point in time, the TOP query by count.
4) To implement the "feedback / loop-back" use a static BlockingQueue in the QueryEventAdapter which the downstream bean populates.
Detailed code below -
1) EPN assembly file
  <wlevs:event-type-repository>
 
        <wlevs:event-type type-name="QueryEvent">
            <wlevs:properties>            
                <wlevs:property name="queryText" type="char" length="200"/>
            </wlevs:properties>
        </wlevs:event-type>
       
        <wlevs:event-type type-name="TimerEvent">
            <wlevs:properties>            
                <wlevs:property name="triggerId" type="bigint"/>
            </wlevs:properties>
        </wlevs:event-type>
       
        <wlevs:event-type type-name="CountEvent">
            <wlevs:properties>
                <wlevs:property name="queryText" type="char" length="200"/>
                <wlevs:property name="queryCount" type="int"/>
            </wlevs:properties>
        </wlevs:event-type>
       
        <wlevs:event-type type-name="QueryOutEvent">
            <wlevs:properties>            
                <wlevs:property name="queryText" type="char" length="200"/>
                <wlevs:property name="triggerId" type="bigint"/>
            </wlevs:properties>
        </wlevs:event-type>
       
    </wlevs:event-type-repository>
<!-- This is the adapter that supplies the input queries. Also, this will update the relation based on feedback from the downstream bean -->
    <wlevs:adapter id="queryEventAdapter" class="com.bea.wlevs.adapter.example.helloworld.QueryEventAdapter" />
<!-- TimerAdapter provides input to TimerStream once every 2 seconds and helps trigger processing -->
    <wlevs:adapter id="timerAdapter" class="com.bea.wlevs.adapter.example.helloworld.TimerAdapter" />
           
<!-- This is the relation that holds all the queries that need to be considered for next round of processing -->   
    <wlevs:channel id="queryEventRelationChannel" event-type="QueryEvent" is-relation="true">
        <wlevs:listener ref="queryEventProcessor"/>
        <wlevs:source ref="queryEventAdapter"/>
    </wlevs:channel>
   
<!-- Triggers processing based on input from TimerAdapter -->
    <wlevs:channel id="timerStream" event-type="TimerEvent">
        <wlevs:listener ref="queryEventProcessor"/>
        <wlevs:source ref="timerAdapter"/>
    </wlevs:channel>
<!-- Contains the CQL that maintains TOP query by count -->   
    <wlevs:processor id="queryEventProcessor" />
   
<!-- HelloWorldBean provides "feedback" to QueryEventAdapter -->
    <wlevs:channel id="countOutputChannel" event-type="CountEvent" advertise="true">
        <wlevs:listener>
            <bean class="com.bea.wlevs.example.helloworld.HelloWorldBean"/>
        </wlevs:listener>
        <wlevs:source ref="queryEventProcessor"/>
    </wlevs:channel>
   
    <wlevs:channel id="queryEventOutputChannel" event-type="QueryOutEvent" advertise="true">
        <wlevs:listener>
            <bean class="com.bea.wlevs.example.helloworld.HelloWorldBean"/>
        </wlevs:listener>
        <wlevs:source ref="queryEventProcessor"/>
    </wlevs:channel>2) TimerAdapter
public void run() {
        // Run this test for 10 seconds
        for (int i=0; i<5; i++) {          
           
            try {
                synchronized (this) {
                    wait(2000);
                }
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
           
            generateMessage();
            triggerId++;
        }
    }
    private void generateMessage() {
        EventType     type   = etr_.getEventType("TimerEvent");
        EventProperty idProp = type.getProperty("triggerId");
        Object        event  = type.createEvent();
       
        idProp.setValue(event, triggerId);
   
        eventSender.sendInsertEvent(event);     
    }3) CQL that maintains TOP query by count
<processor>
    <name>queryEventProcessor</name>
    <rules>
      <view id="R1">
        <![CDATA[              
                    SELECT
                           R.queryText AS queryText,
                           COUNT(*) AS queryCount
                    FROM
                           queryEventRelationChannel AS R
                    GROUP BY
                           queryText                                                           
        ]]>
      </view>
     
      <view id="R2">
        <![CDATA[              
                    SELECT
                           R1.queryText AS queryText,
                           R1.queryCount AS queryCount
                    FROM
                           R1
                    ORDER BY queryCount DESC ROWS 1                                                           
        ]]>
      </view>
     
      <query id="countRule">
        <![CDATA[
                 RSTREAM(
                    SELECT
                         queryText,
                         queryCount
                    FROM
                         timerStream[NOW] as T,
                         R2
                 )
        ]]>
      </query>
     
      <query id="queryRule">
        <![CDATA[
                 RSTREAM(
                    SELECT
                           R.queryText AS queryText,
                           T.triggerId AS triggerId   
                    FROM
                           timerStream[NOW] as T,
                           queryEventRelationChannel AS R                         
                 )
        ]]>
      </query>
    </rules>
   </processor>
  
  <channel>
      <name>countOutputChannel</name>
      <selector>countRule</selector>
  </channel>
  <channel>
      <name>queryEventOutputChannel</name>
      <selector>queryRule</selector>
  </channel>
   4) Downstream bean that provides feedback to the QueryEventAdapter
public void onInsertEvent(Object event) {
         EventType eventType = etr_.getEventType(event);
        if (eventType.getTypeName().equals("HelloWorldEvent")) {           
             String prop = (String)eventType.getPropertyValue(event, "message");
               System.out.println("HelloWorldEvent: " + prop);
        }
        else if (eventType.getTypeName().equals("QueryOutEvent")) {
                String queryText = (String)eventType.getPropertyValue(event, "queryText");
                Long id = (Long)eventType.getPropertyValue(event, "triggerId");
             System.out.println("QueryOutEvent: " + "queryText= " + queryText + " triggerId=" + id);
             
        }
        else if (eventType.getTypeName().equals("CountEvent")) {
             int count = (Integer)eventType.getPropertyValue(event, "queryCount");
             String queryText = (String)eventType.getPropertyValue(event, "queryText");
                System.out.println("CountEvent: " + "queryText= " + queryText + " count=" + count);
                
                // Send the feedback to the QueryEventAdapter - this includes details on the TOP query in the
               //  last round of processing and the associated count
               //
               // The QueryEventAdapter will use this info to update the input relation by removing those many
               // copies of the TOP query
                QueryEventAdapter.deleteEvent(event);
        }
        else {
             System.out.println("No matching event type");
        }
    }5) QueryEventAdapter - supplies the input queries. Updates the input relation based on feedback from downstream bean
private static ArrayBlockingQueue<Object> delList;
// Invoked by downstream bean to provide feedback
public static void deleteEvent(Object event) {
         delList.offer(event);
    }
   
  public void run() {
         initialize();
        generateQueryEvent();
      
    }
   
    private void initialize() {
        type   = etr_.getEventType(QUERY_EVENT_TYPE);
        queryTextProp = type.getProperty(QUERY_TEXT_PROP_NAME);
       
        delList = new ArrayBlockingQueue<Object>(1000);
    }
    private void generateQueryEvent() {      
        String queryText;    
        Object event;
       
        // Wait till t = 1 second
        sleep(1000);
        event = type.createEvent();
        queryText = "11111";
        queryTextProp.setValue(event, queryText);
        eventSender.sendInsertEvent(event);
       
        event = type.createEvent();
        queryText = "22222";
        queryTextProp.setValue(event, queryText);
        eventSender.sendInsertEvent(event);
       
        // Wait till t = 1.5 second
        sleep(500);
        event = type.createEvent();
        queryText = "11111";
        queryTextProp.setValue(event, queryText);
        eventSender.sendInsertEvent(event);
       
        event = type.createEvent();
        queryText = "11111";
        queryTextProp.setValue(event, queryText);
        eventSender.sendInsertEvent(event);  
       
        // HANDLE DELETE EVENTS HERE
        processDeleteEvents(1500);
       
        // Wait till t = 3 second
        event = type.createEvent();
        queryText = "22222";
        queryTextProp.setValue(event, queryText);
        eventSender.sendInsertEvent(event);
       
        event = type.createEvent();
        queryText = "22222";
        queryTextProp.setValue(event, queryText);
        eventSender.sendInsertEvent(event);
       
        // Wait till t = 3.5 second
        sleep(500);
        event = type.createEvent();
        queryText = "22222";
        queryTextProp.setValue(event, queryText);
        eventSender.sendInsertEvent(event);
       
        event = type.createEvent();
        queryText = "11111";
        queryTextProp.setValue(event, queryText);
        eventSender.sendInsertEvent(event);
       
        // HANDLE DELETE EVENTS HERE
        processDeleteEvents(2500);
       
               
    }
   
    private void processDeleteEvents(long timeout) {
         Object delEvent = null;
         
         try {
              System.out.println("Waiting for delete request");
              delEvent = delList.poll(timeout, TimeUnit.MILLISECONDS);
         }
         catch (InterruptedException ie) {
              System.out.println("QueryEventAdapter interrupted .... ");
         }
         if (delEvent != null) {
              System.out.println("Got request for delete");
              // delEvent is an instance of CountEvent
              EventType eventType = etr_.getEventType(delEvent);
              int count = (Integer)eventType.getPropertyValue(delEvent, "queryCount");
              String queryText = (String)eventType.getPropertyValue(delEvent, "queryText");
              
              System.out.println("DelEvent: " + "queryText= " + queryText + " count=" + count);
              // Send as many MINUS elements as the obtained count
              for (int i=0; i<count; i++) {
                   Object minusEvent = type.createEvent();
                   queryTextProp.setValue(minusEvent, queryText);
                       // THIS IS THE API to delete the relevant item from the relation
                   eventSender.sendDeleteEvent(minusEvent);
              }
         }
         else {
              System.out.println("Timed out ...");
         }
    }
   

Related

Problem with PPR event based on Two fields.

Hi,
I am facing an issue in PPR event based on two fields
There are two fields in the page based on its values the third item should be required Yes/No
foriegn & classification are the field attributes
when foriegn is N and classification is CONFIDENTIAL_IND or SECRET_IND
then DCN is the field which should be mandatory
But when I chenge foriegn to Y then also the field DCN is showing mandatory
Please find the issue where i have done worng positng the code which is highlited is the area i am facing issue.
DcnRequired is TransientAttribute
Sample code of the PPR :
AM Method
    public void handleShipmentCreateValidations(){
        OAViewObject vo = (OAViewObject)findViewObject("ShippingRequestCreatePVO1");
        OARow row = (OARow)vo.first();
       
        OAViewObject shipVo = (OAViewObject)findViewObject("ShippingRequestCreateVO1");
        OARow shipRow = (OARow)shipVo.getCurrentRow();
        String reason = (String)shipRow.getAttribute("ShipmentReason");
        String classification = (String)shipRow.getAttribute("Classification");
        String foriegn = (String)shipRow.getAttribute("ForeignShipment");
        Date expected = (Date)shipRow.getAttribute("ExpectedReturnDate");
       String receipt = (String)shipRow.getAttribute("ReceiptBackoutInd");
      
        if (receipt == null || "N".equals(receipt)){
            row.setAttribute("PoRequired","no");
        }
        else if("Y".equals(receipt))
        {
            row.setAttribute("PoRequired","yes");  
        }
       
       
    if ("OTHER".equals(reason))
        {
            row.setAttribute("ShipOtherRender",Boolean.TRUE);
        }
      /*  if (reason == null){
            row.setAttribute("ShipOtherRender",Boolean.FALSE);
        }*/
        else {
            row.setAttribute("ShipOtherRender",Boolean.FALSE);
        }
        *if (("CONFIDENTIAL_IND".equals(classification)||"SECRET_IND".equals(classification))*
        *&& "N".equals(foriegn)) {*
            *row.setAttribute("DcnRequired","yes");*
        *}*
        *if ("N".equals(foriegn) &&*
        *("CONFIDENTIAL_IND".equals(classification)||"SECRET_IND".equals(classification))){*
            *row.setAttribute("DcnRequired","yes");*
        *}*
        *else {*
            *row.setAttribute("DcnRequired","no");*
          
        *}* 
         if (expected == null){
            row.setAttribute("PaperworkNeeded","no");
        }
        else if (expected != null) {
             row.setAttribute("PaperworkNeeded","yes");
        } 
        if (foriegn == null || "N".equals(foriegn)){
              row.setAttribute("ContractRequired","no");
              row.setAttribute("AttentionRequired","no");
              row.setAttribute("PublicDomainRequired","no");
          }
          else if  ("Y".equals(foriegn))
          {
              row.setAttribute("ContractRequired","yes");
              row.setAttribute("AttentionRequired","yes");
              row.setAttribute("PublicDomainRequired","yes");
             
          }
         
    }   CO code
      if ("UpdateValidations".equals(pageContext.getParameter(OAWebBeanConstants.EVENT_PARAM)))   
      {       
        am.invokeMethod("handleShipmentUpdateValidations");   
        } and ppr event name UpdateValidations
Edited by: user1000 on Sep 28, 2010 8:45 AM
Edited by: user1000 on Sep 28, 2010 8:56 AM 
Krishna
        if ("N".equals(foriegn) &&
        ("CONFIDENTIAL_IND".equals(classification)||"SECRET_IND".equals(classification))){
           row.setAttribute("DcnRequired","yes");
        }
        else {
            row.setAttribute("DcnRequired","no");
          
        }Seems same code you have written again.Try to use above code.
One more thing try to debug where your code is flowing when you select foriegn as Y
Thanks
AJ 
Hi Ajay,
Thanks for the reply.
Actually The mistake is with the with the EVENT_NAME which is specified in the item properties.
Now I have reduced the repetition of code as you specified and corrected the event name.
Krishna

about selectInput popup page?

i want to add filter function to the table in the popupp age?
i noticed that tp4 has provide a function to disable the search region,but how can i add filter function to the table? 
Hi,
you can add a query listener to the table. This allows you to specify query conditions: The following code sample is used on a filterable table to ensure the user provided valid search criteria. You should be able to modify it such that you add a new query filter instead of checking the existing for validness
    public void onQuery(QueryEvent queryEvent) {
       
        // pre-processing code here       
       
        boolean invokeQuery = true;
        FilterableQueryDescriptor fqd = (FilterableQueryDescriptor) queryEvent.getDescriptor();
        Map map = fqd.getFilterCriteria();
       
        // ensure DepartmentId contains a Number
        String departmentId = (String) map.get("DepartmentId");
       
        if (departmentId != null && departmentId.length()>0){
           
            try {
              // try to parse String to integer
               Long.parseLong(departmentId);
               
            } catch (Exception ex) {
                // not a string
                System.out.println("Not a string");
                // add some error message here
                // unset selection
                map.remove("DepartmentId");
                invokeQuery = false;
            }
        }
                       
        // ensure the initial character is in uppercase
        String departmentName = (String) map.get("DepartmentName");
        if (departmentName != null && departmentName.length()>0){
            StringBuffer sbuf = new StringBuffer();
            sbuf.append(departmentName.substring(0,1).toUpperCase());
            sbuf.append(departmentName.substring(1));
            map.put("DepartmentName",sbuf.toString());           
        }
        // you can add additional filter criteria here
       
        fqd.setFilterCriteria(map);
        
        if (invokeQuery) {
            invokeMethodExpression("#{bindings.DepartmentsView1Query.processQuery}",
                                                   Object.class,QueryEvent.class,
                                                   queryEvent);
        }
        // post processing code here
       
    }Frank 
thank you Frank,but i'm using inputListOfValues(model driven), the popup page does not created by myself, how can i add a query listener to the table ?

How custom paging works with oracle database

I am working to create a customize paging using oracle database.My code is only gives me the first 20 records from the database and I know because everytime it loads it get the value 20 and it does not show the paging number. I would like to have the paging to be displayed in the first page and the gridview to show the next 20 items every time I change the page nubmer. How can I do this. thanks
Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
           
            BindData()
      
    End Sub
Sub BindData()
        Dim pageSize As Integer = 20
        Dim UpperRowNumber As Integer = ((gvItem.PageIndex + 1) * pageSize) + 1
        Dim LowerRowNumber As Integer = (((gvItem.PageIndex + 1) - 1) * pageSize) + 1
        Label1.Text = gvItem.PageIndex + 1
        Label2.Text = gvItem.PageCount
        Dim connectionString As String = ConnectionStrings("ConnectionString").ConnectionString
        Dim oOracleConn As OracleConnection = New OracleConnection(connectionString)
        oOracleConn.Open()
        Dim strStringBuilder As StringBuilder
        strStringBuilder = New StringBuilder
        With strStringBuilder
            .Append("SELECT b.*, ROWNUM ")
            .Append("FROM (SELECT a.*, func_get_unit_name (unit_name) plan_unit_description, ROWNUM r__ ")
            .Append("FROM (SELECT   (SUBSTR (item, 1, 4) || '.' || SUBSTR (item, 5) ) item_number, ")
            .Append(" idescr short_description, idescrl long_description, ")
            .Append(" iunits unit_name, ispecyr spec_year, iobselet, item ")
            .Append("FROM(itemlist) ")
            .Append("WHERE item <> '2999509/00001' ")
            .Append(" AND iobselet = 'N' ")
            .Append(" AND ispecyr = '05' ")
            .Append(" ORDER BY item) a ")
            .Append(" WHERE ROWNUM < (:rownumberA)) b ")
            .Append(" WHERE r__ >= :rownumberB ")
        End With
        'CREATE A NEW COMMAND AND PASS THE SQL STATEMENT / CONNECTION OBJECT
        Dim cmdItemDetail As OracleCommand = New OracleCommand()
        cmdItemDetail.Parameters.Add(":rownumberA", OracleType.Int32).Value = UpperRowNumber
        cmdItemDetail.Parameters.Add(":rownumberB", OracleType.Int32).Value = LowerRowNumber
        cmdItemDetail.Connection = oOracleConn
        cmdItemDetail.CommandType = CommandType.Text
        cmdItemDetail.CommandText = strStringBuilder.ToString
        ' Dim adItemDetail As New OleDbDataAdapter(cmdItemDetail)
        Dim adItemDetail As New OracleDataAdapter(cmdItemDetail)
        Dim dsItemDetail As New DataSet
        'Fill the dataset with the result of our query from the specified command
        adItemDetail.Fill(dsItemDetail, "ItemDetails")
        'Bind the DataSet to the GridView
        gvItem.DataSource = dsItemDetail
        gvItem.DataBind()
    End Sub<%# Page Language="VB"  MaintainScrollPositionOnPostback="true" AutoEventWireup="false" EnableViewState="false" CodeFile="custPaging.aspx.vb" Inherits="ItemPerLine" %>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html xmlns="http://www.w3.org/1999/xhtml" >
<head>
<title> projects</title>
</head>
<body>
 
    <form id="form1" runat="server">
       
       
        <asp:Label ID="Label1" runat="server" Text="Label"></asp:Label>
       
        <asp:Label ID="Label2" runat="server" Text="Label"></asp:Label>
       
        <asp:DropDownList ID="DropDownList1" runat="server" AutoPostBack="True"
            Height="18px" Width="101px">
            <asp:ListItem>10</asp:ListItem>
            <asp:ListItem>20</asp:ListItem>
            <asp:ListItem>30</asp:ListItem>
        </asp:DropDownList>
       
        <asp:Label ID="lblView" runat="server" ></asp:Label>
                
            <asp:GridView ID="gvItem"
               runat="server"
               CellPadding="4"
               ForeColor="#333333"
               Width="99%"
               HorizontalAlign="Left"
               AutoGenerateColumns="False"
               AllowPaging="True"
               PageSize="20"
               OnPageIndexChanging ="gvItem_PageIndexChanging"
               AllowSorting="True" 
               BackColor="PapayaWhip" Font-Size="Small"
               Font-Bold="False" style="border-top-style: groove; border-right-style: groove; border-left-style: groove; border-bottom-style: groove"  >
              
              
                <FooterStyle BackColor="#5D7B9D" Font-Bold="True" ForeColor="White" />
                <PagerSettings Position="TopAndBottom" />
                <RowStyle BackColor="#F7F6F3" ForeColor="#333333" Font-Size="Small" HorizontalAlign="Left" />
                 <SelectedRowStyle BackColor="#E2DED6" Font-Bold="True" ForeColor="#333333" Font-Size="Small" HorizontalAlign="Left" />
                <PagerStyle BackColor="#FFE0C0" ForeColor="Black" HorizontalAlign="Right"  Font-Size="X-Large" />
                <HeaderStyle BackColor="#5D7B9D" Font-Bold="True" ForeColor="White" Font-Size="Smaller" HorizontalAlign="Left" />
                <AlternatingRowStyle BackColor="White" ForeColor="#284775" />                                        
                <Columns>
                   
                    <asp:BoundField DataField="Item_Number" HeaderText="Line Number" >
                        <HeaderStyle Font-Size="Small" />
                        <ItemStyle Font-Size="Smaller" />
                    </asp:BoundField>
                    <asp:BoundField DataField="Short_Description" HeaderText="Item Number" >
                        <HeaderStyle Font-Size="Small" />
                        <ItemStyle Font-Size="Smaller" />
                    </asp:BoundField>
                    <asp:BoundField DataField="Long_Description" HeaderText="Item Description" >
                        <HeaderStyle Font-Size="Small" />
                        <ItemStyle Font-Size="Smaller" />
                    </asp:BoundField>
                    <asp:BoundField DataField="Unit_Name" HeaderText="Unit_Name" >
                        <HeaderStyle Font-Size="Small" />
                        <ItemStyle Font-Size="Smaller" />
                    </asp:BoundField>
               
                </Columns>
             
            </asp:GridView>
     
       
</form>
</body>
</html> 
Looks like you are on the right track. Just make sure that you are returning the total row count so that you may base your paging on that.
SELECT b.*, ROWNUM
FROM (
      SELECT a.*,
             func_get_unit_name (unit_name) plan_unit_description,
             ROW_NUMBER() OVER(item) R__, --get the rows based on your item order
             count(*) OVER() TotalRows --get the total rows so you may make your page count
        FROM (
                select  
                      (SUBSTR (ITEM, 1, 4) || '.' || SUBSTR (ITEM, 5) ) ITEM_NUMBER,
                      IDESCR SHORT_DESCRIPTION,
                      IDESCRL LONG_DESCRIPTION,
                      IUNITS UNIT_NAME,
                      ISPECYR SPEC_YEAR,
                      IOBSELET,
                      ITEM
                 from ITEMLIST
                where ITEM  '2999509/00001'
                      and IOBSELET = 'N'
                      and ISPECYR = '05'
                order by ITEM
             ) a
  ) B
WHERE R__ between  :ROWNUMBERA and :rownumberB --go ahead and use a between on this
;we tend to create a PAGER control (instead of relying on the grid controls for this)
                    <asp:Repeater ID="pagerCounter" runat="server">
                        <ItemTemplate>
                              <asp:LinkButton ID="Pager" runat="server"  CommandName="pageNumber"   />                       
                        </ItemTemplate>
                    </asp:Repeater> 
...code behind...
    Private Sub CreateManualPager(ByVal PageCounter As Integer)
        Dim pages As New List(Of Integer)
        For i = 1 To PageCounter
            pages.Add(i)
        Next
        pagerCounter.DataSource = pages
        pagerCounter.DataBind()
    End Sub
    Protected Sub RepeaterLoad(ByVal sender As Object, ByVal e As RepeaterItemEventArgs) Handles pagerCounter.ItemDataBound
        Dim data As Integer
        data = CType(e.Item.DataItem, Integer)
        Dim cc As LinkButton = CType(e.Item.FindControl("Pager"), LinkButton)
        cc.CommandName = "PageChange"
        cc.CommandArgument = data.ToString
        cc.Text = data.ToString
        If data - 1 = CurrentPage Then
            cc.Enabled = False
        End If
    End Sub
    Public ReadOnly Property GetStartRow() As Integer
        Get
            GetStartRow = (CurrentPage * gv.PageSize) + 1
        End Get
    End Property
    Public ReadOnly Property GetEndRow() As Integer
        Get
            Return (CurrentPage * gv.PageSize) + gv.PageSize
        End Get
    End Property
    Protected Function EndRow() As Integer
        EndRow = GetEndRow
        If Not _DataSource Is Nothing AndAlso EndRow > _DataSource.TotalRows Then
            EndRow = _DataSource.TotalRows
        End If
        Return EndRow
    End Function
    Private _CurrentPage As Integer
    Public Property CurrentPage() As Integer
        Get
            Return _CurrentPage
        End Get
        Set(ByVal value As Integer)
            _CurrentPage = value
        End Set
    End Property
    Protected Sub pager_ItemCommand(ByVal source As Object, ByVal e As System.Web.UI.WebControls.RepeaterCommandEventArgs) Handles pagerCounter.ItemCommand
        Dim pageSelected As Integer = integer.parse(e.CommandArgument.ToString)
        Dim sDir As String = ""
        CurrentPage = pageSelected - 1
        --> either raise event or rerun & bind your grid here!
    End Subwith that you will need to create the event handler so when someone clicks on the page number it will take that and convert it to your x to (x+20).
this has the downside that on each click you are running the entire query then only returning 20 records (so it this is a long running and heavy query this is not the optimal solution)

Logs not clearing from JE 5.0.97

EDIT: Thank you to anyone who has taken the time to read this posting. Although still not answered, this question is no longer relevant as I have redesigned the program to no longer require the functionality in question. I am using JE 5.09.7 and dealing with an issue while trying to delete pairs from my DB.  Out of the five sections of code below, the one labeled "DB STATUS TEST METHOD" is the code that I have been able to use to confirm that the JE is apparently being told to delete the pairs since the results it prints out show that the pairs have been removed. However I noticed that the log files never go away.  After digging around in both my code and JE’s code I found that within following code from "com.sleepycat.je.cleaner.UtilizationCalculator" none of “obsoleteSize” values for any of the files are ever large enough to cause the file to be marked for removal.  My conclusion from this is that I am either doing something quite wrong or there is a bug in JE that no one has yet to find, the later possibility seeming to be very unlikely. This said, if anyone could please provide some light to my situation I would REALLY appreciate it. I will say in advance that the code I am working with was taken from an older version of the upper level  software that I am using and if anyone recognizes it, please do not say anything. This program is not likely to ever be released to the public as I do not want to support it externally and if it is released all credit will be properly given to the appropriate people and organizations.  EDIT: After further digging it looks like the line below from com.sleepycat.je.cleaner.FileSummary.calculateObsoleteSize(int lnObsoleteSize) always has a "totalLNSize" value that causes the "obsoleteSize" value to always be magnitudes smaller than the value of "summary.totalSize" that is it compared against.  I'm am still trying to figure out where these values are being pulled from, but I am convinced that whatever is causing these values to be so mismatched is the source of this issue.  To be clear, before this code runs, I am using a cursor to delete all of the records from one database after having first copied them to a new database, so the "obosoleteSize" value should be either equal or almost equal to the size of the "totalLNSize" and it is not.  I hope this helps anyone who sees this post understand the issue better.
/* Leftover (non-IN non-LN) space is considered obsolete. */
            final int leftoverSize = totalSize - (totalINSize + totalLNSize);
 Thank you in advance to anyone who takes the time to read this. Jason Corekin
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
JE com.sleepycat.je.cleaner.UtilizationCalculator CODE
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
/* Calculate totals and find the best file. */
Long bestFile = null;
int bestUtilization = 101;
long totalSize = 0;
long totalObsoleteSize = 0;
for (final Map.Entry<Long, FileSummary> entry :
             fileSummaryMap.entrySet()) {
            final Long file = entry.getKey();
            final long fileNum = file.longValue();
            final FileSummary summary = entry.getValue();
            final int obsoleteSize = isProbe ?
                summary.getMaxObsoleteSize() :
                getObsoleteSize(summary);
            /*
             * If the file is already being cleaned, only total the
             * non-obsolete amount.  This is an optimistic prediction of the
             * results of cleaning, and is used to prevent over-cleaning.
             */
            if (cleaner.getFileSelector().isFileCleaningInProgress(file)) {
                final int utilizedSize = summary.totalSize - obsoleteSize;
                totalSize += utilizedSize;
                if (isLoggingLevelFine) {
                    LoggerUtils.logMsg
                        (logger, env, Level.FINE,
                         "Skip file previously selected for cleaning: 0x" +
                         Long.toHexString(fileNum) + " utilizedSize: " +
                         utilizedSize + " " + summary);
                }
                continue;
            }
            /* Add this file's value to the totals. */
            totalSize += summary.totalSize;
            totalObsoleteSize += obsoleteSize;
            /* Don't select an explicitly excluded file. */
            if (excludeFiles.contains(file)) {
                continue;
            }
            /* Skip files that are too young to be cleaned. */
            if (fileNum > lastFileToClean) {
                continue;
            }
            /* Select this file if it has the lowest utilization so far. */
            final int thisUtilization =
                FileSummary.utilization(obsoleteSize, summary.totalSize);
            if (bestFile == null || thisUtilization < bestUtilization) {
                bestFile = file;
                bestUtilization = thisUtilization;
            }
            /* Return all low utilization files. */
            if (lowUtilizationFiles != null &&
                thisUtilization < useMinUtilization) {
                lowUtilizationFiles.add(file);
            }
}
/*
         * The first priority is to clean the log up to the minimum utilization
         * level, so if we're below the minimum (or an individual file is below
         * the minimum for any file), then we clean the lowest utilization
         * (best) file.  Otherwise, if there are more files to migrate, we
         * clean the next file to be migrated.  Otherwise, if cleaning is
         * forced (for unit testing), we clean the lowest utilization file.
         */
final Long fileChosen;
final String loggingMsg;
final int totalUtilization =
            FileSummary.utilization(totalObsoleteSize, totalSize);
if (totalUtilization < useMinUtilization ||
            bestUtilization < useMinFileUtilization) {
            fileChosen = bestFile;
            loggingMsg = "Chose lowest utilized file for cleaning.";
} else if (!isBacklog &&
                   filesToMigrate.hasNext(fileSummaryMap)) {
            fileChosen = filesToMigrate.next(fileSummaryMap);
            loggingMsg = "Chose file from files-to-migrate for cleaning.";
} else if (forceCleaning) {
            fileChosen = bestFile;
            loggingMsg = "Chose file for forced cleaning (during testing).";
} else {
            fileChosen = null;
            loggingMsg = "No file selected for cleaning.";
        }
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
DB STATUS TEST METHOD
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
    void printDBNames() {
        Database curDB;
        List<String> dbNames = env.getDatabaseNames();
        System.out.println("The current list of Database Names is:");
        for (String dbName : dbNames) {
            System.out.println(dbName);
            TransactionUsers.getInstance().putTransaction(env.beginTransaction(null, trConfig), "HelloLucene:printDBNames:line 552");
            curDB = env.openDatabase(TransactionUsers.getInstance().getTransaction(), dbName, dbConfig);
            System.out.println("The current number of pairs in " + dbName + " is:\n" + curDB.count());
            curDB.close();
            TransactionUsers.getInstance().getTransaction().abort();
            TransactionUsers.getInstance().removeTransaction();
        }
    }
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
/////Taken from the programs main class where this code currently sits so that it is easier to get to
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
        envConfig = new EnvironmentConfig();
        dbConfig = new DatabaseConfig();
        trConfig = new TransactionConfig();
        envConfig.setTransactional(true);
//        envConfig.setLocking(true);
        envConfig.setLoggingHandler(logFileHandler);
        envConfig.setAllowCreate(true);
        envConfig.setConfigParam("je.log.nDataDirectories", dataDirNumber);
//        envConfig.setConfigParam("je.txn.deadlockStackTrace", "true");
        envConfig.setConfigParam("je.cleaner.minUtilization", "90");
        envConfig.setConfigParam("je.cleaner.expunge", "false");
        envConfig.setConfigParam("je.env.runCleaner", "false");
        envConfig.setConfigParam("je.cleaner.minAge", "1");
        envConfig.setConfigParam("je.cleaner.fetchObsoleteSize", "true");
        envConfig.setConfigParam("je.cleaner.adjustUtilization", "false");
        envConfig.setCacheMode(CacheMode.MAKE_COLD);
 
 
 
//        envConfig.setConfigParam("je.txn.dumpLocks", "true");
//        envConfig.setLockTimeout(4500000, TimeUnit.MILLISECONDS);
        //When working with so much data it's good to give the program as much
        //cache to work with as possible
        envConfig.setCachePercent(75);
        //It's also good to write in larger blocks to reduce the file count used
        //to store the DBs.
        envConfig.setConfigParam("je.log.fileMax", "100000000");
        dbConfig.setTransactional(true);
        dbConfig.setAllowCreate(true);
        dbConfig.setSortedDuplicates(false);
//        dbConfig.setNodeMaxEntries(32767);
        trConfig.setSync(true);
        env = new Environment(indexDir, envConfig);
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////CODE USED TO CALL clewanLog() PROGRAMATICALLY
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
 
env.compress();
boolean anyCleaned = false;
CheckpointConfig force = new CheckpointConfig();
force.setForce(true);
env.checkpoint(force);
while (env.cleanLog() > 0) {
anyCleaned = true;
}
if (anyCleaned) {
env.checkpoint(force);
}
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
 
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
//////Class that does the work to call the JE delete code.
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////////////////
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.util.Random;
import java.util.concurrent.locks.ReentrantLock;
import com.sleepycat.je.Cursor;
import com.sleepycat.je.Database;
import com.sleepycat.je.DatabaseEntry;
import com.sleepycat.je.DatabaseException;
import com.sleepycat.je.LockMode;
import com.sleepycat.je.OperationStatus;
import com.sleepycat.je.Transaction;
/**
* Port of Andi Vajda's DbDirectory to Java Edition of Berkeley Database
*
*/
public class File extends Object {
    static protected Random random = new Random();
    protected DatabaseEntry key, data;
    protected long length, timeModified;
    protected String name;
    protected byte[] uuid;
    private static long cursorDeleteCounter = 0, indexDeleteCounter = 0, blocksDeleteCounter = 0;
    private static final ReentrantLock lock = new ReentrantLock();
    protected File(String name) throws IOException {
        setName(name);
        data = new DatabaseEntry(new byte[32]);
    }
    protected File(JEDirectory directory, String name, boolean create)
            throws IOException {
        this(name);
        if (!exists(directory)) {
            if (!create) {
                throw new IOException("File does not exist: " + name);
            } else {
                DatabaseEntry key = new DatabaseEntry(new byte[24]);
                DatabaseEntry data = new DatabaseEntry(null);
                Database blocks = directory.blocks;
                Transaction txn = TransactionUsers.getInstance().getTransaction();
                data.setPartial(true);
                uuid = new byte[16];
                try {
                    do {
                        /* generate a v.4 random-uuid unique to this db */
                        random.nextBytes(uuid);
                        uuid[6] = (byte) ((byte) 0x40 | (uuid[6] & (byte) 0x0f));
                        uuid[8] = (byte) ((byte) 0x80 | (uuid[8] & (byte) 0x3f));
                        System.arraycopy(uuid, 0, key.getData(), 0, 16);
                        // TODO check LockMode
                    } while (blocks.get(txn, key, data, null) != OperationStatus.NOTFOUND);
                } catch (DatabaseException e) {
                    e.printStackTrace();
                    throw e;
                }
            }
        } else if (create) {
            length = 0L;
        }
    }
    protected String getName() {
        return name;
    }
    private void setName(String name) throws IOException {
        ByteArrayOutputStream buffer = new ByteArrayOutputStream(128);
        DataOutputStream out = new DataOutputStream(buffer);
        out.writeUTF(name);
        out.close();
        key = new DatabaseEntry(buffer.toByteArray());
        this.name = name;
    }
    protected byte[] getKey() throws IOException {
        if (uuid == null) {
            throw new IOException("Uninitialized file");
        }
        return uuid;
    }
    protected boolean exists(JEDirectory directory) throws IOException {
        Database index = directory.index;
        Transaction txn = TransactionUsers.getInstance().getTransaction();
        try {
            // TODO check LockMode
            if (index.get(txn, key, data, LockMode.DEFAULT) == OperationStatus.NOTFOUND) {
                return false;
            }
        } catch (DatabaseException e) {
            throw new IOException(e.getMessage());
        }
        byte[] bytes = data.getData();
        ByteArrayInputStream buffer = new ByteArrayInputStream(bytes);
        DataInputStream in = new DataInputStream(buffer);
        length = in.readLong();
        timeModified = in.readLong();
        in.close();
        uuid = new byte[16];
        System.arraycopy(bytes, 16, uuid, 0, 16);
        return true;
    }
/////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////
/////////METHOD THAT DOES ALL THE WORK TO CALL THE JE DELETE CODE
/////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////
    protected void delete(JEDirectory directory) throws IOException {
        if (!exists(directory)) {
            throw new IOException("File does not exist: " + getName());
        }
        Cursor cursor = null;
        Database blocks = null, index = null;
        Transaction txn = TransactionUsers.getInstance().getTransaction();
        long cursorDeleteCounterLocal = 0 , indexDeleteCounterLocal = 0,blocksDeleteCounterLocal = 0;
        try {
            try {
                byte[] bytes = getKey();
                int ulen = bytes.length + 8;
                byte[] cursorBytes = new byte[ulen];
                DatabaseEntry cursorKey = new DatabaseEntry(cursorBytes);
                DatabaseEntry cursorData = new DatabaseEntry(null);
                index = directory.index;
                blocks = directory.blocks;
                System.arraycopy(bytes, 0, cursorBytes, 0, bytes.length);
                cursorData.setPartial(true);         
                cursor = blocks.openCursor(txn, null);
                if (cursor.getSearchKey(cursorKey, cursorData, null) != OperationStatus.NOTFOUND) {
                    if(OperationStatus.SUCCESS == cursor.delete()){
                        cursorDeleteCounterLocal++;
                    }
                    advance: while (cursor.getNext(cursorKey, cursorData, null) != OperationStatus.NOTFOUND) {
                        byte[] temp = cursorKey.getData();
                        for (int i = 0; i < bytes.length; i++) {
                            if (bytes[i] != temp[i]) {
                                break advance;
                            }
                        }
                        if(OperationStatus.SUCCESS == cursor.delete()){
                            cursorDeleteCounterLocal++;
                        }
                    }
                }         
            }finally {
                if (cursor != null) {
                    cursor.close();
                }
                if(blocks != null){
                    if(OperationStatus.SUCCESS == blocks.delete(txn, key)){
                        blocksDeleteCounterLocal++;
                    }
                }
                if(index != null){
                    if(OperationStatus.SUCCESS == index.delete(txn, key)){
                        indexDeleteCounterLocal++;
                    }
                }
         
                try{
                    lock.lock();
                    indexDeleteCounter+=indexDeleteCounterLocal;
                    blocksDeleteCounter+=blocksDeleteCounterLocal;
                    cursorDeleteCounter+=cursorDeleteCounterLocal;
                }finally{
                    lock.unlock();
                }
            }
        } catch (DatabaseException e) {
            throw new IOException(e.getMessage());
        }
    }
    public static long getIndexDeleteCounter(){
        return indexDeleteCounter;
    }
    public static long getBlocksDeleteCounter(){
        return blocksDeleteCounter;
    }
    public static long getCursorDeleteCounter(){
        return cursorDeleteCounter;
    }
}
/////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////
Hi Jason,         envConfig.setConfigParam("je.env.runCleaner", "false");  Do you see the logs getting cleaned if you set this to "true"? When investigating how the cleaner works you may want to have a look at the cleaner methods available under EnvironmentStats: http://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/EnvironmentStats.html Thanks,Bogdan
Bogdan, Unfortunately a quick double check of the work that I had don earlier this week resulted in the same results, envConfig.setConfigParam("je.env.runCleaner", "true"); produces no effect.  I will run things over more in the morning, but based on everything that I have seen putting this value back to default only produces a thread that waits until it is cleaned up, they never do any work.  This is based off of analysis gathered using the VisualVM java profiler. That said, I did find a setting that does make a difference.  If I force the situation with:  envConfig.setConfigParam("je.cleaner.forceCleanFiles", "0-ffffffff");, the program does start to either delete or rename the files based of the value of the "je.cleaner.expunge" setting.  If I combine this change with the extraction of archive.env.cleanLog() from the code below and then comment out that code, I do see the program start to maintain the log files more closely to what I would expect.  But I do not think that this setting should be necessary, nor am I convinced that it is safe. boolean anyCleaned = false;while (archive.env.cleanLog() > 0) {anyCleaned = true;}if (anyCleaned) {CheckpointConfig force = new CheckpointConfig();force.setForce(true);archive.env.checkpoint(force);} I will look at the link you sent me tomorrow, thanks for sending it. Thanks for replying, I look forward to any other ideas you or anyone else might have. Jason
I'm sorry if I am being too persistent, but I found some more information that I thought might be helpful.  Immediately Below is all of the configuration parameters that I am using to setup JE.  Below that is the contents of a log file that I found “je.info.0” that has the utilization values that I was talking about.  Again, I am sorry if I am posting too much I just want to help anyone who is trying to help me as much as possible.  Jason Corekin
        envConfig = new EnvironmentConfig();
        dbConfig = new DatabaseConfig();
        trConfig = new TransactionConfig();
        envConfig.setTransactional(true);
//        envConfig.setLocking(true);
        envConfig.setLoggingHandler(logFileHandler);
        envConfig.setAllowCreate(true);
        envConfig.setConfigParam("je.log.nDataDirectories", dataDirNumber);
//        envConfig.setConfigParam("je.txn.deadlockStackTrace", "true");
        //envConfig.setConfigParam("je.cleaner.minUtilization", "90");
        envConfig.setConfigParam("je.cleaner.expunge", "false");
        envConfig.setConfigParam("je.env.runCheckpointer", "false");
        envConfig.setConfigParam("je.env.runCleaner", "false");
        envConfig.setConfigParam("je.cleaner.minAge", "1");
        envConfig.setConfigParam("je.cleaner.fetchObsoleteSize", "true");
        envConfig.setConfigParam("je.cleaner.adjustUtilization", "false");
        envConfig.setConfigParam("je.cleaner.readSize", "209715200");
        envConfig.setConfigParam("je.log.iteratorMaxSize", "419430400");
        envConfig.setConfigParam("je.cleaner.forceCleanFiles", "0-ffffffff");
      
        envConfig.setCacheMode(CacheMode.MAKE_COLD);
//        envConfig.setConfigParam(EnvironmentConfig.FILE_LOGGING_LEVEL, "ALL");
//        envConfig.setConfigParam(EnvironmentConfig.CONSOLE_LOGGING_LEVEL,
//                                 "ALL");
      
      
        Logger parent = Logger.getLogger("com.sleepycat.je");
        parent.setLevel(Level.FINE);  // Loggers will now publish more
                                      // detailed messages.
      
      
//        envConfig.setConfigParam("je.txn.dumpLocks", "true");
//        envConfig.setLockTimeout(4500000, TimeUnit.MILLISECONDS);
        //When working with so much data it's good to give the program as much
        //cache to work with as possible
        envConfig.setCachePercent(75);
        //It's also good to write in larger blocks to reduce the file count used
        //to store the DBs.
        envConfig.setConfigParam("je.log.fileMax", /*"1073741824"*/ "104857600");
        dbConfig.setTransactional(true);
        dbConfig.setAllowCreate(true);
        dbConfig.setSortedDuplicates(false);
//        dbConfig.setNodeMaxEntries(32767);
        trConfig.setSync(true);
        env = new Environment(indexDir, envConfig);
        analyzer = new StandardAnalyzer(Version.LUCENE_46);
                                   ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////LOG FILE////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////                            
2013-12-20 14:55:39.352 UTC INFO [M:\ArchiveServerTestspace\TESTDB] Chose file from files-to-migrate for cleaning. fileChosen: 0x0 (adjustment disabled) totalUtilization: 99 bestFileUtilization: 99 isProbe: false
2013-12-20 14:55:40.631 UTC INFO [M:\ArchiveServerTestspace\TESTDB] CleanerRun 1 ends on file 0x0 probe=false invokedFromDaemon=false finished=true fileDeleted=false nEntriesRead=9336 nINsObsolete=246 nINsCleaned=0 nINsDead=0 nINsMigrated=0 nBINDeltasObsolete=0 nBINDeltasCleaned=0 nBINDeltasDead=0 nBINDeltasMigrated=0 nLNsObsolete=2579 nLNsCleaned=6507 nLNsDead=6502 nLNsMigrated=5 nLNsMarked=0 nLNQueueHits=2 nLNsLocked=0 logSummary=<CleanerLogSummary endFileNumAtLastAdjustment="0x6" initialAdjustments="1" recentLNSizesAndCounts=""> inSummary=<INSummary totalINCount="246" totalINSize="446393" totalBINDeltaCount="0" totalBINDeltaSize="0" obsoleteINCount="246" obsoleteINSize="446393" obsoleteBINDeltaCount="0" obsoleteBINDeltaSize="0"/> estFileSummary=<summary totalCount="9443" totalSize="104848565" totalINCount="246" totalINSize="446393" totalLNCount="9086" totalLNSize="104398920" maxLNSize="16434" obsoleteINCount="246" obsoleteLNCount="2579" obsoleteLNSize="131578" obsoleteLNSizeCounted="2579" getObsoleteSize="581223" getObsoleteINSize="446393" getObsoleteLNSize="131578" getMaxObsoleteSize="581223" getMaxObsoleteLNSize="131578" getAvgObsoleteLNSizeNotCounted="NaN"/> recalcFileSummary=<summary totalCount="9443" totalSize="104848565" totalINCount="246" totalINSize="446393" totalLNCount="9086" totalLNSize="104398920" maxLNSize="0" obsoleteINCount="246" obsoleteLNCount="2579" obsoleteLNSize="131578" obsoleteLNSizeCounted="2579" getObsoleteSize="581223" getObsoleteINSize="446393" getObsoleteLNSize="131578" getMaxObsoleteSize="581223" getMaxObsoleteLNSize="131578" getAvgObsoleteLNSizeNotCounted="NaN"/> lnSizeCorrection=NaN newLnSizeCorrection=NaN estimatedUtilization=99 correctedUtilization=99 recalcUtilization=99 correctionRejected=false
2013-12-20 14:55:40.631 UTC INFO [M:\ArchiveServerTestspace\TESTDB] Chose file from files-to-migrate for cleaning. fileChosen: 0x1 (adjustment disabled) totalUtilization: 99 bestFileUtilization: 99 isProbe: false
2013-12-20 14:55:41.598 UTC INFO [M:\ArchiveServerTestspace\TESTDB] CleanerRun 2 ends on file 

WebCenter Portal Analytics - Page Metrics issue

Working with the analytics service for a webcenter Framework (11.1.1.7.1) application I have found that to track metrics, one must include manually the code to send the metrics to the analytics collector. Following this document: 46.2.3.1 Including Event Code for Page Views Oracle States that the following code: <af:resource type="javascript">
  function initPageLoadEvent(event) {
  AdfCustomEvent.queue(event.getSource(),
  "generatePageEvent",
  {pageName:"Page and Login Statistics"},
  true);
  event.cancel();
  }
  </af:resource>
<af:clientListener method="initPageLoadEvent" type="load"/>
<af:serverListener type="generatePageEvent"
  method="#{AnalyticsInstrumentation.sendPageEvent}"/>
  Has to be included between the <af:document> tag, on every page of the portal. The metrics bean has been configured correctly, and page metrics get sent out using a command button for testing purpouses ont the page template: <af:outputText id="js_wcanalytics" escape="false"
                             value='<script type="text/javascript">function initPageLoadEvent(event) {console.log("DEBUG-EVENT:initPageLoadEvent");AdfCustomEvent.queue(event.getSource(),"generatePageEvent",{pageName:"#{pageDocBean.title}"},true);event.cancel();}</script>'/>
              <af:commandButton id="btn_test_analytics"
                                text="Test - WC Analytics">
                <af:clientListener method="initPageLoadEvent" type="click"/>
                <af:serverListener type="generatePageEvent"
                                   method="#{AnalyticsInstrumentation.sendPageEvent}"/>
              </af:commandButton> 
 The issue i am asking for help, is to avoid modifying every single page of the portal (aproximately 1000) to include the client and server listeners to fire on the document load event. Is there a way to include the analytics page metric code on webcenter pagetemplates code?
So far I haven't found a feasible solution to avoid adding the code to each an every page on the portal. Altough I have found a variation to what the documentation says about the javascript analytics function, that prevents to modify the pageName parammeter that passes to the AnaliticsInstrumentation.sendPageEvent Method. The code is: <f:facet name="metaContainer">
<af:outputText id="js_wcanalytics" escape="false" value='<script type="text/javascript">function initPageLoadEvent(event) {AdfCustomEvent.queue(event.getSource(),"generatePageEvent",{pageName:"#{pageDocBean.title}"},true);event.cancel();}</script>'/>
</f:facet>
<af:clientListener method="initPageLoadEvent" type="load"/>
<af:serverListener type="generatePageEvent" method="#{AnalyticsInstrumentation.sendPageEvent}"/>
 The code has to be added on every page, inside the <af:document> tag. A relatively quick way to add the code to every page is to export all the metadata files and using a text editor, search for the <af:form> tag and inject the code above the before the <af:form> start tag. Some problems with this solution: Analytics event cannot be disabled. clientListener nor serverLister has the rendered property, so there is no way to pass a pagetemplate or portal attribute to temporally disable Analytics tracking. these listeners have to be mandatory included as first children of the <af:document> tag, because the type="load" event on the client listener, only applies to the <af:document> component.  Other posible solutions:Use a servlet filter that sends the page metrics to the analytics collectorUse a PhaseListener that fires on the afterPhase when the page is fully loaded to send the page metrics to the analytics collector. If someone has a way to execute the analytics script on the pageTemplate side, that would be an optimal solution, that avoids the editing of every single page.
Hi. I prefer a solution based on a Custom View Handler that is registered in faces-config.xml and affects all the Portals. Let me check my old repositories and I can give you a cleaner solution which will nto require to edit any page . Kind regards.
My solution: <application>
    <default-render-kit-id>oracle.adf.rich</default-render-kit-id>
    <view-handler>your.pigeon.sexy.package.MyCustomViewHandler</view-handler>
  </application>
The Java Class:public class MyCustomViewHandler extends CustomViewHandler {
    /**
    * Response time
    */
    private static int RESPONSETIME = 100;
    /**
    * Class name for logging system
    */
    private static final String CLASS_NAME = MyCustomViewHandler.CLASS_NAME;
    /**
    * Logging
    */
    private static final ADFLogger LOG =
        ADFLogger.createADFLogger(MyCustomViewHandler.class);
    /**
    * Default constructor
    * #param viewHandler - ViewHandler
    */
    public MyCustomViewHandler(ViewHandler viewHandler) {
        super(viewHandler);
    }
    /**
    * Override targetViewId for include sendPageViewToAnalytics event
    * #param pCustomPortalNavigation
    * #param pSiteStructureResource
    * #param pTransferable
    * #param pBoolean
    * #param pScope
    * #return ViewId
    */
    #Override
    protected String getPageTargetViewId(CustomPortalNavigation pCustomPortalNavigation,
                                        SiteStructureResource pSiteStructureResource,
                                        Transferable pTransferable,
                                        boolean pBoolean, Scope pScope) {
        String viewId =
            super.getPageTargetViewId(pCustomPortalNavigation, pSiteStructureResource,
                                      pTransferable, pBoolean, pScope);
        sendPageViewToAnalytics(viewId);
        return viewId;
    }
    /**
    * Send pageView to Analytics Event Collector
    * #param pViewId
    */
    public static void sendPageViewToAnalytics(String pViewId) {
        final String METHOD_NAME = "sendPageViewToAnalytics";
        LOG.entering(CLASS_NAME, METHOD_NAME);
        if (StringUtils.isBlank(pViewId)) {
            LOG.fine(CLASS_NAME, METHOD_NAME, "Encountered blank viewId");
            return;
        }
        if (!AnalyticsUtil.isSendingEvents()) {
            LOG.info(CLASS_NAME, METHOD_NAME,
                    "Analytics is not sending events.");
            return;
        }
        String resourceId = pViewId;
        String viewId = pViewId;
        // Retrieve page path from viewId
        String sSearchString = "/oracle/webcenter/portalapp/pages/";
        int pagePathStart = pViewId.indexOf(sSearchString);
        if (pagePathStart > -1) {
            pagePathStart += sSearchString.length();
            viewId = pViewId.substring(pagePathStart);
        }
        int extensionIndex = viewId.indexOf(".jspx");
        if (extensionIndex > -1) {
            viewId = viewId.substring(0, extensionIndex);
        }
        HttpServletRequest request = JSFUtils.getRequest();
        final String requestUser = request.getRemoteUser();
        LOG.fine(CLASS_NAME, METHOD_NAME, "Event user: " + requestUser);
        LOG.fine(CLASS_NAME, METHOD_NAME, "Event pageName: " + viewId);
        AnalyticsUtil.sendPageViewEvent(resourceId,
                                            PortalInfo.WC_APPLICATION_NAME,
                                            requestUser, RESPONSETIME, viewId,
                                            false, request);
            LOG.fine(CLASS_NAME, METHOD_NAME,
                    "Page event sent for resourceId: " + resourceId);
        LOG.exiting(CLASS_NAME, METHOD_NAME);
    }
}
Thanks a lot for your response. Will try it out.
Your solution works well in certain cases. For example, I tried to capture the PrettyUrl to register as the viewId, because there are somre links that users access directly through the jspx page, and others with the PrettyURL format from the navigation model, and when I triet to capture the PrettyUrl on the ViewHandler, there was a NullPointerException while validating the Current navigation node during page refresh; maybe because the event on the ViewHandler get called before the NavigationModel gets initialized. So, based on your solution, I created the following PhaseListener:  import javax.faces.context.FacesContext;
import javax.faces.event.PhaseEvent;
import javax.faces.event.PhaseId;
import javax.faces.event.PhaseListener;
import javax.servlet.http.HttpServletRequest;
import oracle.adf.share.logging.ADFLogger;
import oracle.webcenter.framework.service.AnalyticsUtil;
import oracle.webcenter.framework.service.JSFUtils;
import oracle.webcenter.portalframework.sitestructure.SiteStructure;
import oracle.webcenter.portalframework.sitestructure.SiteStructureContext;
import oracle.webcenter.portalframework.sitestructure.SiteStructureResource;
import oracle.webcenter.portalframework.sitestructure.SiteStructureUtils;
public class AnalyticsPhaseListener implements PhaseListener {
    private static final long serialVersionUID = 1L;
    /**
     * Response time
     */
    private static int RESPONSETIME = 100;
    /**
     * Class name for logging system
     */
    private static final String CLASS_NAME = AnalyticsPhaseListener.CLASS_NAME;
    /**
     * Logging
     */
    private static final ADFLogger LOG =
        ADFLogger.createADFLogger(AnalyticsPhaseListener.class);
    public AnalyticsPhaseListener() {
        super();
    }
   /**
     * Phase Listener para la fase RENDER_RESPONSE
     * Que se encarga de enviar el evento de PageView
     * al Colector de Analiticas de WebCenter Portal
     * #param phaseEvent
     */
    public void afterPhase(PhaseEvent phaseEvent) {
        final String METHOD_NAME = "afterPhase";
        LOG.entering(CLASS_NAME, METHOD_NAME);
        // Checks whether Analytics events must be sent
        if (!AnalyticsUtil.isSendingEvents()) {
            LOG.fine(CLASS_NAME, METHOD_NAME,
                     "Analytics is not sending events.");
            return;
        }
        if (phaseEvent.getPhaseId() == PhaseId.RENDER_RESPONSE) {
            LOG.fine(CLASS_NAME, METHOD_NAME,
                     "Entering PhaseId.RENDER_RESPONSE");
            //Obtener Faces Context
            FacesContext context = FacesContext.getCurrentInstance();
            //Se obtiene el ViewId que corresponde al archivo jspx de la pagina
            String analyticsViewId = context.getViewRoot().getViewId();
            //Se Obtiene el pageName de Evaluar la expresion #{pageDocBean.title}
            String pageName =
                (String)JSFUtils.resolveExpression("#{pageDocBean.title}");
            //En caso que se acceda a la pagina usando PrettyUrl del modelo de navegacion, se captura el ViewID
            //Desde el modelo de navegacion
            //Obtener Modelo de Navegacion del Portal
            SiteStructure model =
                SiteStructureContext.getInstance().getCurrentModel();
            if (model != null) {
                SiteStructureResource resource = model.getCurrentSelection();
                if (resource != null) {
                    if(resource.isCurrentlySelected()){
                    analyticsViewId =
                            SiteStructureUtils.encodeUrl(resource.getPrettyUrl());
                   
                    //TODO Track viewId of SiteStudio Resource Pages
                   
                    }                   
                }
            }
            LOG.fine(CLASS_NAME, METHOD_NAME,
                     "analyticsViewId = " + analyticsViewId);
            LOG.fine(CLASS_NAME, METHOD_NAME, "pageName = " + pageName);
            //Se obtiene el Usueario que visita la pagina del HTTPServletRequest
            HttpServletRequest request =
                (HttpServletRequest)context.getExternalContext().getRequest();
            String requestUser = request.getRemoteUser();
            //Registrar eventos para usuarios anonymous
            if (requestUser == null) {
                requestUser = "anonymous";
            }
            LOG.fine(CLASS_NAME, METHOD_NAME, "requestUser = " + requestUser);
            //Enviar eventos a Webcenter Analytics si se han capturado correctamente los parametros
            if (analyticsViewId == null || analyticsViewId.isEmpty() ||
                requestUser == null) {
                LOG.fine(CLASS_NAME, METHOD_NAME,
                         "Analytics not sending events. analyticsViewId IS NULL!");
                return;
            }
            if (pageName == null || pageName.isEmpty() ||
                requestUser == null) {
                LOG.fine(CLASS_NAME, METHOD_NAME,
                         "Analytics not sending events. pageName IS NULL!");
                return;
            }
            LOG.fine(CLASS_NAME, METHOD_NAME,
                     "Sending PageViewEvent for: " + "[RESOURCEID_ = " +
                     analyticsViewId + "]" + "[NAME_ = " + pageName + "]" +
                     "[USER_ = " + requestUser + "]");
            AnalyticsUtil.sendPageViewEvent(analyticsViewId, "HomeMinhacienda",
                                            requestUser, 0, pageName, false,
                                            request);
            LOG.fine(CLASS_NAME, METHOD_NAME,
                     "PageViewEvent SENT! for analyticsViewId = " +
                     analyticsViewId);
        }
        LOG.exiting(CLASS_NAME, METHOD_NAME);
    }
    public void beforePhase(PhaseEvent phaseEvent) {
    }
    public PhaseId getPhaseId() {
        return PhaseId.RENDER_RESPONSE;
    }
}
 Register the PhaseListener on the faces-config.xml file: <lifecycle>
    <phase-listener>oracle.webcenter.skin.view.SkinPhaseListener</phase-listener>
    <phase-listener>my.package.AnalyticsPhaseListener</phase-listener>
  </lifecycle>
 Deploy and test. The events gets succsefully tracked on each page, after the pages is rendered. Thankyou very much for your help.  PDTA: In case for the webcenter application, i don't think there is possible to overwrite the default ViewHandler; but I know that you can add PhaseListeners using the WebCenterExtensionLibrary.
Hi. Your solution will register PPR and other JSF actions at it will invoke the RENDER_RESPONE phase.If you're facing issues with my solution may I can take a look into why are you not being able to take the prettyUrl propertly.The ViewHandler will be only executed once when page is rendering not like a Phase Listener in RENDER_RESPONSE. Kind
You're Right navigating throug tabs invokes the phaseListener Phase. Nullpointer, isung the ViewHandler ocurrs Here (Line 63): #Override
    protected String getPageTargetViewId(CustomPortalNavigation pCustomPortalNavigation,
                                        SiteStructureResource pSiteStructureResource,
                                        Transferable pTransferable,
                                        boolean pBoolean, Scope pScope) {
        String viewId =
            super.getPageTargetViewId(pCustomPortalNavigation, pSiteStructureResource,
                                      pTransferable, pBoolean, pScope);
        sendPageViewToAnalytics(viewId);
        return viewId;
    }
    /**
    * Send pageView to Analytics Event Collector
    * #param pViewId
    */
    public static void sendPageViewToAnalytics(String pViewId) {
        final String METHOD_NAME = "sendPageViewToAnalytics";
        LOG.entering(CLASS_NAME, METHOD_NAME);
        if (pViewId == null || pViewId.isEmpty()) {
            LOG.fine(CLASS_NAME, METHOD_NAME, "Encountered blank viewId");
            return;
        }
        if (!AnalyticsUtil.isSendingEvents()) {
            LOG.info(CLASS_NAME, METHOD_NAME,
                    "Analytics is not sending events.");
            return;
        }
        String resourceId = pViewId;
        String viewId = pViewId;
        // Retrieve page path from viewId
        String sSearchString = "/oracle/webcenter/portalapp/pages/";
        int pagePathStart = pViewId.indexOf(sSearchString);
        if (pagePathStart > -1) {
            pagePathStart += sSearchString.length();
            viewId = pViewId.substring(pagePathStart);
        }
        int extensionIndex = viewId.indexOf(".jspx");
        if (extensionIndex > -1) {
            viewId = viewId.substring(0, extensionIndex);
        }
        FacesContext context = FacesContext.getCurrentInstance();
      
        SiteStructure model =
            SiteStructureContext.getInstance().getCurrentModel();
        if (model != null) {
            SiteStructureResource resource = model.getCurrentSelection();
              if (resource != null && resource.isCurrentlySelected()) {
                resourceId =
                        SiteStructureUtils.encodeUrl(resource.getPrettyUrl());
                viewId = resource.getTitle();
            }
        }
      
        HttpServletRequest request =
            (HttpServletRequest)context.getExternalContext().getRequest();
        String requestUser = request.getRemoteUser();
        //Registrar eventos para usuarios anonymous
        if (requestUser == null) {
            requestUser = "anonymous";
        }
        //SEND PAGE VIEW EVENT
        //Capturar parametro de pageName del Evento
        /*String pageName = (String)event.getParameters().get("pageName");
            if (pageName == null || pageName.isEmpty() || requestUser == null)
                return;*/
        //final String requestUser = request.getRemoteUser();
        LOG.fine(CLASS_NAME, METHOD_NAME, "Event user: " + requestUser);
        LOG.fine(CLASS_NAME, METHOD_NAME, "Event pageName: " + viewId);
        AnalyticsUtil.sendPageViewEvent(resourceId, "HomeMinhacienda",
                                        requestUser, RESPONSETIME, viewId,
                                        false, request);
        LOG.fine(CLASS_NAME, METHOD_NAME,
                "Page event sent for resourceId: " + resourceId);
        LOG.exiting(CLASS_NAME, METHOD_NAME);
    }
 Will try to debug why there is no navigation resource when the page is Refreshed.
Client decided to go with the solution proposed by oracle: adding the javascript Function and client/server listeners on every page. using WLST exportMetadata() and importMetadata() commands, and a text processor, I could easily add the following lines, right before the <af:form> tag inside the <af:document> <f:facet name="metaContainer">
     <c:set var="analyticsViewId"
          value="#{(empty navigationContext.currentNavigationModel.currentSelection.PrettyUrl or not navigationContext.currentNavigationModel.currentSelection.currentlySelected) ? facesContext.viewRoot.viewId : navigationContext.currentNavigationModel.currentSelection.PrettyUrl}"
          xmlns:c="http://java.sun.com/jsp/jstl/core"/>
     <c:set var="pageName"
          value="#{pageDocBean.title}"
          xmlns:c="http://java.sun.com/jsp/jstl/core"/>
     <af:outputText id="js_wcanalytics" escape="false" value='<script type="text/javascript">function initPageLoadEvent(event) {AdfCustomEvent.queue(event.getSource(),"generatePageEvent",{analyticsViewId:"#{analyticsViewId}",pageName:"#{pageName}"},true);event.cancel();}</script>'/>
</f:facet>
<af:clientListener method="initPageLoadEvent" type="load"/>
<af:serverListener type="generatePageEvent" method="#{AnalyticsInstrumentation.sendPageEvent}"/>
 Luckyly, the clientListener does not interfere with Jquery and other JS libraries on the page. Your solution would be the ideal one due to the effort that takes to implement. Thanks a lot for your assistance
Thanks for sharing your thoughts and solution. Kind regards.

Categories

Resources