Oracle Soa Suite Unit test with Groovy (11g, 12c) - part 1

It's going to be a little long but if you stay until the last line, you are going to know what i did to create dynamic xml requests and responses usign the unit test framework of the Soa Suite with groovy. (yes, groovy!)

I'm going to give some background of the problem and later my own personal solution. Lets begin :D

[Update 30/06/2016] Wrote a second blog about subject with more functionality for the tests; JUnit test execution, pre load of dat and post processing of information. http://carlgira.blogspot.com.es/2016/06/oracle-soa-suite-unit-test-with-groovy.html

PROBLEM

I was trying to create Unit Test for some Bpel with several Web Service calls, human task, JCAs to database, and i get really frustated trying to create dynamic requests or responses using the tool within the Jdeveloper.

I wanted to re-create some fields, update dates etc.

The only thing i found was something that the TestSuites supports but the graphic wizards dont show. You can use small Xpath functions to replace values of the payloads in your TestSuite.

The next image shows the initiation message of a TestCase. You can see that after the payload, there is an element called "update". This element only receives two attributes, the "updateLocation" that refers a XpathLocation of a field to update, and the "updateXpathFunction" with the xpath function with the new value.

You could think that this could work but there is a problem with the Xpath functions you can use, there are only avalaible the "basic" Xpath functions https://www.w3.org/TR/1999/REC-xpath-19991116/#corelib ( Node-Set, String, Boolean and math functions - check the link with the full list)

But there are also some diferences between versions of the Soa Suite that i check:

Soa Suite 11.1.1.7: Only basic Xpath functions
Soa Suite 11.1.1.7.3: After this versions they add several functions of the http://www.oracle.com/XSL/Transform/java/oracle.tip.pc.services.functions.Xpath20 that you can use, (all of them related with dates and durations).
Soa Suite 12.1.3: I check again if there was more supported functions in 12c but it seems that again you only can use the basic Xpath functions. (lost all the Xpath20 functions dont know why)

So, there are few options to create dynamic XML in your unit test.

SOLUTION

I love the way the SoapUI gives you the possibility to create dynamic request using incrusted groovy inside of your XML. So i've spent some time looking for doing this possible with the Soa Suite.

I tried for long to found the "spot" of code that i should replace to make this possible, and i found it :D. The basic idea is that you can create dynamic XML request or response using incrusted groovy.

The best way to explain what it does is with an example.

•  When you build the request add some code of groovy. In the next example you can see how a date is created and later a random integer value.

When you execute the test, those values are replaced and you get a dynamic XML!!

HOW IT WORKS

I had to look for the spot with minimun changes. I found it in the class oracle.integration.platform.testfwk.TestCase in the fabric-ext.jar

I create an exact copy of that class and add some code to execute the groovy code inside the method "populatePayload".

Finally, the idea is to add my jar to the server classpath and make sure that is loaded first that the original one.

You can see everything inside the sources

INSTALLATION/CONFIGURATION

These are all the instructions to make it work.

• Download the sources from github https://github.com/carlgira/soa-unit-test/tree/master/custom-test-case

• Create the jar (Soa 11.1.1.7 or 12.1.3, use a custom profile for your version)
• mvn -Dsoa-version=11.1.1.7 clean package
• mvn -Dsoa-version=12.1.3 clean package

• Copy the file custom-test-case-1.0-jar-with-dependencies.jar to the directory Middleware\Oracle_SOA1\soa\modules\oracle.soa.fabric_11.1.1 in 11g or in the Middleware/soa/soa/modules/oracle.soa.fabric_11.1.1 on 12c.

• Add the custom-test-case-1.0-jar-with-dependencies.jar in the classpath of the MANIFEST file of the jar oracle.soa.fabric.jar. Make sure that this file appears before than the fabric-ext.jar in the classpath variable. (create a backup of jar before the modification)

•  Reboot your server

•  Now you can add to your XML messages in the Jdeveloper the groovy code :D

SUMMARY

• The utility allow you to create dynamic XML messages with groovy.
• You need to change the classpath to make sure to load the modified class.
• This utility is intended to your test environment.

Thanks! i hope someone find this useful.

Full Weblogic Custom Authenticator (11g, 12c)

I could not find a full example of custom weblogic asserter using maven. I got some examples using ant, but i dont like that... it never works for me for some reason, so i simply create a maven project using the sources of [1] getting some ideas of this github snippet wiht the "WebLogic MBean Maker" pom.xml [2], and most important the explanation on the creation of a custom authenticator in the book "Securing WebLogic Server 12c" chapter 4  (really recommended if you want to understand what you are doing).

There are lots of blogs with the same sources (SimpleSampleIdentityAsserterProviderImpl), so you are going to find the same example over and over again on internet, based in a really old sample used with Bea-server. In some moment in time the authenticator sources came with a sample web application to test it but you could not find that application anymore.

I don't want to extend to much with this, so i'll try to remark only the most important, and give you the basic instructions to make it work.

• First understand the difference between and authenticator and a asserter. The asserter is a way to "translate" a key or token to a set of credentials, that can be used to authenticate (lets say a cookie, http header or a certificate). The authenticator has the responsibility to check if with those credentials  the user can continue to the protected resource. (A better explanation can be found in the A-team blog about asserters [5])

          Why this is important? Because on many internet sources you are only going to get the asserter and not the full authenticator (the asserter plus the LoginModule). If you only get the asserter you are half way. Check [6] to get the LoginModule or this one [7] to extend your already mbean asserter to an authenticator provider.

• The second thing was to test it. It was funny, it was harder to find the right configuration for the web application than the sources of the authenticator. 

          With the authenticator we create a custom token. The key that the user must use, so the asserter gets activated.  A token can be a cookie, a http header or a certificate [8], so to test your application you must send the token the correct way (i use the cookie and the http header and both works fine)

        You also need to configure your web application with the protected resources, the roles and the principal mapping.

        In your web.xml add something like this (add the "login-config", the "security-role" and a "security-constraint")

 
  CLIENT-CERT
 

 
  LoggedUsers
 

 
  
   Protected resources
   protected/*
   GET
   POST
  
  
   LoggedUsers
  
 

       In your weblogic.xml add the mapping between the role and the principals.

        
  LoggedUsers
  users
 

**** For a beautiful example with the sources of the web application see [9]

CONFIGURE

The project is on github  https://github.com/carlgira/soa-utils/tree/master/http-token-authenticator

• In the pom.xml configure the "mdw.home" with your Middleware path (on the 11g or 12c profile)
• According with your version probably you'll have to change some jar paths.
• Execute maven install (11.1.1.7 or 12.1.3)
• mvn -Dsoa-version=12.1.3 clean install
• mvn -Dsoa-version=11.1.1.7 clean install
• Copy the jar to the path Middleware\wlserver_10.3\server\lib\mbeantypes
• Reboot your server
• Go to Security Realms->myrealm->Providers and create a SimpleSampleIdentityAsserter.
•  Make sure to put all the authenticators flags to "SUFFICENT"
• Reorder your authenticators and put the new one the last.
• Reboot your server

TEST A PROTECTED APPLICATION

I use the firebug add-on of Firefox to test it. Just create a custom cookie named "PerimeterAtnToken" and value "username=weblogic".

You can also make a http request with a http header named "PerimeterAtnToken" and with a value of "username=weblogic".

That should be enough, if not, make sure to check the "References" read the book and all the examples.

Thanks!

REFERENCES

1. Simple Sample Custom Identity Asserter for Weblogic Server 12c http://weblogic-wonders.com/weblogic/2014/01/13/simple-sample-custom-identity-asserter-weblogic-server-12c/
2. WebLogic MBean Maker. https://gist.github.com/kares/356576
3 Creating a wlfullclient.jar. https://docs.oracle.com/cd/E12840_01/wls/docs103/client/jarbuilder.html
4. Securing WebLogic Server 12c,
5. Why do I need an Authenticator when I have an Identity Asserter?, Oracle A-team http://www.ateam-oracle.com/why-do-i-need-an-authenticator-when-i-have-an-identity-asserter/
6. Do You Need to Develop a Custom Authentication Provider?http://docs.oracle.com/cd/E21764_01/web.1111/e13718/atn.htm#DEVSP220
7. Weblogic Identity Asserter and Athorization Provider in one! http://darylwiest.blogspot.com.es/2015/02/weblogic-identity-asserter-and.html
Extends the mbean to an authenticator
8. Passing Tokens for Perimeter Authentication http://docs.oracle.com/cd/E21764_01/web.1111/e13718/ia.htm#DEVSP254
9. SiteMinder WebLogic Security Provider Mock, https://gibaholms.wordpress.com/2015/01/21/siteminder-weblogic-security-provider-mock/
10. Mock Weblogic Login module - Identity Asserter and Authenticator,  http://danielveselka.blogspot.com.es/2012/04/mock-weblogic-login-module-identity.html

DeadLock Detector

DeadLock Detector

Two utilities to detect a deadlock inside a JVM.

See https://github.com/carlgira/soa-utils/tree/master/deadlock

 

 deadlock-detection

A remote deadlock detector. It connects to a JVM using JMX to detect if a deadlock is ocurring. It needs the host of the JVM and the JMX port.

  $ cd deadlock-detection
  $ mvn clean package
  $ java -jar target/deadlock-detection-1.0-SNAPSHOT.jar localhos:3333

deadlock-detector-service

A Weblogic Rest service to detect if is ocurring a deadlock in the server. It can deployed on Weblogic or tested with spring-boot

• Test with Spring-boot

  $ cd deadlock-detector-service
  $ mvn clean package
  $ java -jar target/deadlock-detector-service-1.0.0.war
  $ curl -X GET http://localhost:8080/deadlock

• Weblogic

  $ cd deadlock-detector-service
  $ mvn clean package
  $ Deploy to Weblogic 
  $ curl -X GET http://localhost:7001/deadlock-detector-service-1.0.0/deadlock

Thanks! 

Soa Worklist Authentication with LdapX509Asserter

Recently I had to configure two way SSL authentication to the worklist, validating the client certificate with one saved in the Ldap.

To do this is necessary to configure the LdapX509Asserter and the LdapAuthenticator. I put all the instructions I had to follow to complete this configuration.

Create PKS12

Have or create the bundle with the certificates necessaries in pks12 format (also called pfx or just p12) to add to Weblogic

You need to add to the bundle the server certificate and the private key.

     openssl pkcs12 -name alias -export -in mycert.crt -inkey mykey.key > server.p12

Create KeyStores

Weblogic saves all the certificates in two keystores

• TrusKeyStore: Save here all the CAs
• IdentityKeyStore: Save here all the certificates and private keys

Create empty TrustKeyStore

     keytool -genkey -alias TrustAlias -keyalg RSA -keystore TrustKeyStore.jks

Import the certificates to the keystores

Importar CA a TrustKeyStore

     keytool -importcert -file my-ca-file.crt-keystore TrustKeyStore.jks -storepass TrustKeyStorePassPhrase

Import pk12 bundle to IdentityStore

     keytool -importkeystore -srckeystore server.p12 -srcstoretype pkcs12  -srcalias alias -destkeystore IdentityKeyStore.jks  -deststoretype jks -deststorepass password -destalias alias

See [2] for Common SSL commands. 

Weblogic Configuration

Transport

Follow instructions in [1] in "Transport" section.

1. Enable "SSL Listener Port" in server General tab.
2. Configure the identity and trust keystores. Sever -> Configuration -> KeyStores. Select "Custom Identity and Java Standard Trust", configure all values.
3. Specify the certificate to use for weblogic using the alias of the certificate used in the IdentityKeyStore. Go to Server -> Configuration -> SSL, configure the alias and the password of IdentityKeyStore
4. Configure Weblogic to "Two way Client Cer Behavior" Sever -> Configuration -> SSL -> Advanced,  choose "Client Cert Requested and Enforced"

Authentication

Follow instruction to configure LdapX509Asserter [3].

• Configure the ldap connection data
• User name attribute in LDAP 
• User filter attributes, the fields to match between the certificate and the LDAP entry. (check the certificate metadata and your ldap

Follow instruction to configure LdapAuthenticator [4]

• Configure the ldap connection
• Configure the user section
• Configure the group section

(Read this article [1],  it is very helpful to understand about authenticators and asserters)

The configuration of the asserter and the authenticator is not hard You need to know the data to connect to the LDAP and the information to identify the user

About those instructions there are couple of important things

1. It's important to know that the IdentityAsserters needs to work with the Authenticators. So it's necessary to configure first the LdapX509Asserter and next the LdapAuthenticator.
2. Make sure to "reorder" the asserters and authenticator in your realm. Put them in the order you want they be executed. (Security Realms -> your realm -> Providers -> Reorder)
3. If you have more than one Authenticator configure the flag "Control Flag" to "SUFFICENT" in "all" the authenticators. 
4. In the LdapX509Authenticator there is a field to configure where is the certificate. It could be "usercertificate" or "usercertificate;binary". I got a problem because my ldap browser shows me the field as "usercertificate;binary" but after testing i had to change it to "usercertificate".

Configure Custom Authenticator

Besides the configuration in the console of the LdapAuthenticator, you have change the property "idstore.type"  in the jps-config.xml.

You can change it manually in jps-config.xml [5], or using the Enterprise Manager going to,  Farm -> Weblogic Domain, right click on your domain, Security -> Security Provider Configuration. In the section "Identity Store Provider" click on "Configure" add the property "idstore.type" according with your provider

The possible values are OID ,  OVD , IPLANET , ACTIVE_DIRECTORY , EDIRECTORY , OPEN_LDAP  [6]

Update Https port in Human Task

For any already deployed workflow task detail applications, change the workflow task display URL to use the correct protocol and port number.

Go to the enterprise manager and open the afected composite. In the "Component Metrics" section click over the human task, "administration" tab, and update the https port [9]

Debug

There two things you could do to debug.

1. Activate the logger of the Authenticators. Go to Server -> Debug. Activate the logger  weblogic->security->atn->DebugSecurityAtn. (This is very useful) [7]
2. Use the next flags  -Dssl.debug=true -Dweblogic.StdoutDebugEnabled=true -Dweblogic.security.SSL.verbose=true   [8]

Test

To test that your LdapX509Asserter is working install the client certificates on your browser and open the worklist on the ssl port.

The browser it's going to show you the list of certificates that you could use to authenticate. If the certificate validates against the server and is equals to the one in the ldap you are inside the worklist.

Thanks!

References

        1. Two way SSL Configuration,   https://www.captechconsulting.com/blogs/two---way-ssl-configuration-for-web-applications

        2. Common SSL commands https://www.sslshopper.com/article-most-common-java-keytool-keystore-commands.html

       3. Configuring an LDAP X509 Identity Assertion Provider https://docs.oracle.com/cd/E13222_01/wls/docs81/secmanage/providers.html#1197612

4. Configure LdapAuthenticator https://docs.oracle.com/middleware/1213/wls/SECMG/ldap_atn.htm#SECMG175

5. Configure custom authenticators http://docs.oracle.com/cd/E28280_01/core.1111/e10043/idstoreadm.htm#JISEC9738

6. Supported LDAPhttp://docs.tpu.ru/docs/oracle/en/fmw/11.1.1.6.0/core.1111/e10043/devauthn.htm

7. Troubleshooting with authenticators https://docs.oracle.com/cd/E24628_01/doc.121/e36415/sec_troubleshoot.htm#EMSEC12981

8. SSL TroubleShooting and Debugging https://blogs.oracle.com/WebLogicServer/entry/ssl_troubleshooting_and_debugg 
9. Managing the URI of the Human Task. http://docs.oracle.com/cd/E23943_01/admin.1111/e10226/hwf_mang.htm#SOAAG3757

Oracle Bussiness Rules - Java API - BucketSet Creation

Hi, i made some test using the Oracle Business Rules java API. There is few documentation about OBR java API, so i think in sharing the code. 

You can use the example for three things:

 1. Creation of a list of values (LOV) BucketSet
 2  Creation of range BucketSet
 3. Loading dictionaries using custom finder class. (Probably the most important)

About the third one, over the internet i just found one example to load a dictionary and was using the DecisionPointDictionaryFinder. That don't work when you have one dictionary linked to another,  so i have to do my own implementation the ListRuleFinder (i recommend using it)

test-bs-rules

 Inside of the project https://github.com/carlgira/soa-utils/tree/master/test-bs-rules

src:
    LovFormBSUtil: List of values BucketSet creation class
    RangeFormBSUtil: Range of values BucketSet creation class

   ListRuleFinder: Custom dictionary finder
   DictionaryUtil: Class to load a dictionary from disk, and a function to validate a dictionary.

resources:
   Two linked and empty dictionaries.
test:
   BucketSetTest: Tests
                           - Load of linked dictionaries using  ListRuleFinder
                           - Creation of a list of values BucketSet
                           - Creation of a range BucketSet


The output it's going to be in the target directory. To execute the maven project you need to configure the oracle.mdw variable.

(Tested in a 11g installation )

Hazelcast Object Inspector

 I made a tiny class to inspect the objects inside the data grid of hazelcast. You can see all the objects in the distributed collection (maps, queues, list, etc).

Just download the sources from github https://github.com/carlgira/utils/tree/master/hazelcast-object-inspector add to the pom.xml the necessary dependencies from your application (if not you are going to get ClassNotFound exception when the application is trying to deserialize your objects).

Customize as you want the class.

Hope this will be helpful

Filter Weblogic Logs

This blog explains how to use a library that i develop, that captures and redirect information from the weblogic logs to any other medium.

The idea was to redirect the information of the proxy services log of the OSB (Oracle Service BUS), but it can be expanded to redirect logs of any kind of application that writes over the Weblogic server logs.

Note: Last year i wrote a blog about this same issue OSB Custom Proxy Logging  , but last time i used  log4j as default logging for weblogic and also use a superior version that the one installed on the server. One colleague told me to find a solution using as default the JDK logging and also not to change the version of the log4j on the server, to avoid problems with support . That's what i achieve with this.

Weblogic Custom Logging

This tool adds a custom handler to the Weblogic Server logger. That handler manages a list of Log4jManagers that filters the logRecords. I add two LogManagers to the lib; one that filters all the OSB messages and redirect them to a file, and another logManager that it's able to write a log for every Proxy Service configured.(You can also add your own custom logManagers to filter other applications)

To configure it, it's necessary to copy the library to the domain/lib, create and configure a startupClass in Weblogic, and finally add a log4j file with the configuration of the appenders and loggers.

The parameters needed to configure the startUpClass are:

1. Log4j.xml location
2. List of active LogHandlers

    /home/carlgira/soa-utils/weblogic-custom-logging/src/test/resources/log4j.xml   com.carlgira.weblogic.loggging.log4j.managers.MyLogManager1
com.carlgira.weblogic.loggging.log4j.managers.MyLogManager2
com.carlgira.weblogic.loggging.log4j.managers.MyLogManager3


You can download the source code from github.
https://github.com/carlgira/soa-utils


These are the detailed instructions:

1. Download and Compile the code

Download the maven project from github. You'll  have to change two things inside of the pom.xml

     1.1 Change the "mdw.path" variable with your environment path. 

     1.2 I test the tool with a 10.3.6 Weblogic, so maybe you'll have to change the name of the com.bea.core.logging jar.

2. Installation

2.1 Copy the generated file to the base_domain/lib folder.
2.2 Edit and copy your own log4j.xml file. In the project there is an example in src/test/resources

3. Create the startupClass

     3.1 In the Weblogic console go to "Startup and Shutdown Classess"

    3.2 Create a new Startup Class. Complete the parameters, and select the targets.
 
     Name: WeblogicCustomLog
     Class Name: com.carlgira.weblogic.loggging.WeblogicCustomLogging




     3.2  Go back to the to the StartupClass and edit the arguments. This time i will configure the OSBProxyLogManager.

Arguments:
 /home/carlgira/soa-utils/weblogic-custom-logging/src/test/resources/log4j.xml com.carlgira.weblogic.loggging.log4j.managers.OSBLogManager







     3.3 Save the changes

4. Create the Weblogic Filter

The tool only captures the logRecords and redirect them to other files. You'll have to manually create the log filters so the information won't be written on the weblogic logs.

     4.1 Go to your domain configuration and click on the "Log Filter" tab




    4.2 Create a Log Filter and edit them to add an expression. For the OSB i use a "MATCHES" operator.

   Message Attribute: MESSAGE
   Operator: MATCHES
   Value: (\[OSB\sTracing\]|\[Rastreo\sde\sOSB\])



    4.3 At the end the expression will look like this.



      4.4 Save all the changes
      4.5 Now it's time to apply the filter to the server. Go to the configuration of the server to the logging tab.




      4.6 Scroll down until the "message destination" and choose the WeblogicCustomFilter.



    4.7 Save all the changes
    4.8 Do this for every server you need to configure.

5. Check everything is ok

After all the changes, reboot your weblogic (the one that you configure as target in the StartupClass). If everything is ok, you will see a trace "weblogic-custom-logging, started".

6. LogManagers

LogManagers have to deliver the LoggerName. You can create a static LoggerName like is the case of the OSBLogManager or a dynamic one, like the OSBProxyLogManager does.

One thing that is VERY IMPORTANT is that those LoggerNames MUST be in the log4j.xml file so the tool knows where to write the filtered messages.

6.1 OSBLogManager

This is a simple class. It only has two attributes, one is the regex that filters the messages and the other one is the static loggerName.

You have to be sure to leave the "OSBServerLogger" in the log4j.xml file.

6.2 OSBProxyLogManager

This class has a more complicated behavior . It gets dynamically the name of the proxy service from the log message, but it must save the loggerName in the MDC of log4j (Is a temporary memory location to save shared variables inside of the executing thread) because not all the OSB proxy log messages brings the proxy name.

To make it work, you'll have to add manually the full name of the proxy service to the log4j.xml file, and replace every "/" for a "-" because the log4j cant detect those characters.




Log4j.xml File



That's all.

Thanks!