Oracle Soa Suite Unit test with Groovy (11g, 12c) - part 2

A month ago I wrote a blog about how to create dynamic xml request with groovy (http://carlgira.blogspot.com.es/2016/05/oracle-soa-suite-unit-test-with-groovy.html i recommend to read that blog first). Since that, i'been working to make it better.

The idea is that you can use groovy code inside the XMLs of a test case to create dynamic values like dates, numbers, alphanumeric (you can do anything you want with groovy), but i wanted one more thing.

PROBLEM

For testing you probably have some pre and post conditions steps, probably you are going to need to share some information between service calls, and maybe you are going to need to get or assign a value to a field from outside the test case.

SOLUTION

The idea was to create a little framework that helps to create more robust test for the SOA Suite, executing and configuring the test case from a JUnit test. The solution allows to:

1. Create dynamic xml request with groovy.
2. Pre-load information to groovy for the execution of the test case.
3. Get a saved value from groovy to the java client, for post-processing or validation.

With these points you can get two scenarios. 

SAMPLE 1 - Simple Test Case (Execution of groovy code)

In the first scenario you get every variable loaded from the XML, (you don't pre-load any information or share fields between requests). all the information of the test case is on the XML request with the groovy code.

Groovy code on Xml request	Xml generated

SAMPLE 2 - Complex Test Case (Execution of groovy code, pre-load of data, saved data for post-processing)

The second example is more complex

Groovy code on Xml request	Generated XML

• A variable "name" is called but is not initialized in groovy. We are going to set that variable before start of the Test in Java with the value "outName"
• The variable "token" is initialized and is saved so it can be used or obtained later for post processing.

HOW IT WORKS

Those samples are executed from a Java client with the desired configuration.

The first sample just execute the test case as it is, the second one has a little more logic that i'm going to explain.

Java Code of Test Sample 2	Test results

1. The first thing is the definition of the names of the composite, test suite and test case that is going to be executed.
2. Before the execution of Test, the variable "name" is set with value "outName". (like a pre-step for the test)
3. Execution of Test.
4. The variable "token" that is set inside of the XML can be obtained after the execution of the test. You can see the value "74" for the token variable in the generated XML and in the test result. (like a post-step for the test)

I'm using two classes inside of the Java Test.

• GroovyShellService: The class has some utilities to create/delete a groovy shell and also functions to get/set variables inside a groovy shell of a test case. 
• UnitTestManager: It has some functions to execute test suites or test cases for a specific composite.

CODE

In Github you are going to find four projects

https://github.com/carlgira/soa-unit-test

• custom-test-case: Lib used to  to resolve the groovy code in the Xml requests.
• soa-unit-test-webapp: War with REST services to manage the creation/deletion/execution of the groovy shells for every test case.
• soa-unit-test-client: Lib with functions to call the REST services and also a utility class to call the test-suites and test-cases from the SOA suite.
• sample-unit-test-project: A simple class with the execution of the two test cases presented on this blog. Also has the sample SOA app for 11g and 12c so you can test it.

After installation the only thing you need to learn is how to use the two classes inside the soa-unit-test-client so you can build your own test cases. Check the sample inside sample-unit-test-project to know how to use them.

INSTALLATION/CONFIGURATION

 

You need to package, install and deploy the four projects.

• Follow all the instructions from http://carlgira.blogspot.com.es/2016/05/oracle-soa-suite-unit-test-with-groovy.html to install custom-test-case in Weblogic server.
• Open the pom.xml from soa-unit-test-client, and sample-unit-test-project and modify the variable of the "mdw.home" in the profile according with your version (11.1.1.7 or 12.1.3) and put the path to your SOA installation.
• Install in the maven repository the soa-unit-test-client. (11.1.1.7 or 12.1.3)
• mvn clean install -Dsoa-version=11.1.1.7 -Dmaven.test.skip=true
• mvn clean install -Dsoa-version=12.1.3 -Dmaven.test.skip=true
• Package soa-unit-test-webapp.
• mvn clean package -Dmaven.test.skip=true
• Deploy the generated war soa-unit-test-webapp in Weblogic.
• Install the SOA-app using the Enterprise manager from sample-unit-test-project/src/test/resources (11.1.1.7 or 12.1.3)
• soa11.1.1.7/sca_soa-test-project_rev1.0.jar
• soa12.1.3/sca_soa-test-project_rev1.0.jar
  
• Open the project sample-unit-test-project and in the class "SoaUnitTest" modify in the constructor the host, port, username and password. 
• In sample-unit-test-project  execute the tests (takes like 2 minutes)
• mvn clean test -Dsoa-version=11.1.1.7
• mvn clean test -Dsoa-version=12.1.3
• Check the two instances created and check that the two test cases are executed correctly.

Thats all,  thanks and i hope someone find this useful.

Oracle Soa Suite Unit test with Groovy (11g, 12c) - part 1

It's going to be a little long but if you stay until the last line, you are going to know what i did to create dynamic xml requests and responses usign the unit test framework of the Soa Suite with groovy. (yes, groovy!)

I'm going to give some background of the problem and later my own personal solution. Lets begin :D

[Update 30/06/2016] Wrote a second blog about subject with more functionality for the tests; JUnit test execution, pre load of dat and post processing of information. http://carlgira.blogspot.com.es/2016/06/oracle-soa-suite-unit-test-with-groovy.html

PROBLEM

I was trying to create Unit Test for some Bpel with several Web Service calls, human task, JCAs to database, and i get really frustated trying to create dynamic requests or responses using the tool within the Jdeveloper.

I wanted to re-create some fields, update dates etc.

The only thing i found was something that the TestSuites supports but the graphic wizards dont show. You can use small Xpath functions to replace values of the payloads in your TestSuite.

The next image shows the initiation message of a TestCase. You can see that after the payload, there is an element called "update". This element only receives two attributes, the "updateLocation" that refers a XpathLocation of a field to update, and the "updateXpathFunction" with the xpath function with the new value.

You could think that this could work but there is a problem with the Xpath functions you can use, there are only avalaible the "basic" Xpath functions https://www.w3.org/TR/1999/REC-xpath-19991116/#corelib ( Node-Set, String, Boolean and math functions - check the link with the full list)

But there are also some diferences between versions of the Soa Suite that i check:

Soa Suite 11.1.1.7: Only basic Xpath functions
Soa Suite 11.1.1.7.3: After this versions they add several functions of the http://www.oracle.com/XSL/Transform/java/oracle.tip.pc.services.functions.Xpath20 that you can use, (all of them related with dates and durations).
Soa Suite 12.1.3: I check again if there was more supported functions in 12c but it seems that again you only can use the basic Xpath functions. (lost all the Xpath20 functions dont know why)

So, there are few options to create dynamic XML in your unit test.

SOLUTION

I love the way the SoapUI gives you the possibility to create dynamic request using incrusted groovy inside of your XML. So i've spent some time looking for doing this possible with the Soa Suite.

I tried for long to found the "spot" of code that i should replace to make this possible, and i found it :D. The basic idea is that you can create dynamic XML request or response using incrusted groovy.

The best way to explain what it does is with an example.

•  When you build the request add some code of groovy. In the next example you can see how a date is created and later a random integer value.

When you execute the test, those values are replaced and you get a dynamic XML!!

HOW IT WORKS

I had to look for the spot with minimun changes. I found it in the class oracle.integration.platform.testfwk.TestCase in the fabric-ext.jar

I create an exact copy of that class and add some code to execute the groovy code inside the method "populatePayload".

Finally, the idea is to add my jar to the server classpath and make sure that is loaded first that the original one.

You can see everything inside the sources

INSTALLATION/CONFIGURATION

These are all the instructions to make it work.

• Download the sources from github https://github.com/carlgira/soa-unit-test/tree/master/custom-test-case

• Create the jar (Soa 11.1.1.7 or 12.1.3, use a custom profile for your version)
• mvn -Dsoa-version=11.1.1.7 clean package
• mvn -Dsoa-version=12.1.3 clean package

• Copy the file custom-test-case-1.0-jar-with-dependencies.jar to the directory Middleware\Oracle_SOA1\soa\modules\oracle.soa.fabric_11.1.1 in 11g or in the Middleware/soa/soa/modules/oracle.soa.fabric_11.1.1 on 12c.

• Add the custom-test-case-1.0-jar-with-dependencies.jar in the classpath of the MANIFEST file of the jar oracle.soa.fabric.jar. Make sure that this file appears before than the fabric-ext.jar in the classpath variable. (create a backup of jar before the modification)

•  Reboot your server

•  Now you can add to your XML messages in the Jdeveloper the groovy code :D

SUMMARY

• The utility allow you to create dynamic XML messages with groovy.
• You need to change the classpath to make sure to load the modified class.
• This utility is intended to your test environment.

Thanks! i hope someone find this useful.

Full Weblogic Custom Authenticator (11g, 12c)

I could not find a full example of custom weblogic asserter using maven. I got some examples using ant, but i dont like that... it never works for me for some reason, so i simply create a maven project using the sources of [1] getting some ideas of this github snippet wiht the "WebLogic MBean Maker" pom.xml [2], and most important the explanation on the creation of a custom authenticator in the book "Securing WebLogic Server 12c" chapter 4  (really recommended if you want to understand what you are doing).

There are lots of blogs with the same sources (SimpleSampleIdentityAsserterProviderImpl), so you are going to find the same example over and over again on internet, based in a really old sample used with Bea-server. In some moment in time the authenticator sources came with a sample web application to test it but you could not find that application anymore.

I don't want to extend to much with this, so i'll try to remark only the most important, and give you the basic instructions to make it work.

• First understand the difference between and authenticator and a asserter. The asserter is a way to "translate" a key or token to a set of credentials, that can be used to authenticate (lets say a cookie, http header or a certificate). The authenticator has the responsibility to check if with those credentials  the user can continue to the protected resource. (A better explanation can be found in the A-team blog about asserters [5])

          Why this is important? Because on many internet sources you are only going to get the asserter and not the full authenticator (the asserter plus the LoginModule). If you only get the asserter you are half way. Check [6] to get the LoginModule or this one [7] to extend your already mbean asserter to an authenticator provider.

• The second thing was to test it. It was funny, it was harder to find the right configuration for the web application than the sources of the authenticator. 

          With the authenticator we create a custom token. The key that the user must use, so the asserter gets activated.  A token can be a cookie, a http header or a certificate [8], so to test your application you must send the token the correct way (i use the cookie and the http header and both works fine)

        You also need to configure your web application with the protected resources, the roles and the principal mapping.

        In your web.xml add something like this (add the "login-config", the "security-role" and a "security-constraint")

 
  CLIENT-CERT
 

 
  LoggedUsers
 

 
  
   Protected resources
   protected/*
   GET
   POST
  
  
   LoggedUsers
  
 

       In your weblogic.xml add the mapping between the role and the principals.

        
  LoggedUsers
  users
 

**** For a beautiful example with the sources of the web application see [9]

CONFIGURE

The project is on github  https://github.com/carlgira/soa-utils/tree/master/http-token-authenticator

• In the pom.xml configure the "mdw.home" with your Middleware path (on the 11g or 12c profile)
• According with your version probably you'll have to change some jar paths.
• Execute maven install (11.1.1.7 or 12.1.3)
• mvn -Dsoa-version=12.1.3 clean install
• mvn -Dsoa-version=11.1.1.7 clean install
• Copy the jar to the path Middleware\wlserver_10.3\server\lib\mbeantypes
• Reboot your server
• Go to Security Realms->myrealm->Providers and create a SimpleSampleIdentityAsserter.
•  Make sure to put all the authenticators flags to "SUFFICENT"
• Reorder your authenticators and put the new one the last.
• Reboot your server

TEST A PROTECTED APPLICATION

I use the firebug add-on of Firefox to test it. Just create a custom cookie named "PerimeterAtnToken" and value "username=weblogic".

You can also make a http request with a http header named "PerimeterAtnToken" and with a value of "username=weblogic".

That should be enough, if not, make sure to check the "References" read the book and all the examples.

Thanks!

REFERENCES

1. Simple Sample Custom Identity Asserter for Weblogic Server 12c http://weblogic-wonders.com/weblogic/2014/01/13/simple-sample-custom-identity-asserter-weblogic-server-12c/
2. WebLogic MBean Maker. https://gist.github.com/kares/356576
3 Creating a wlfullclient.jar. https://docs.oracle.com/cd/E12840_01/wls/docs103/client/jarbuilder.html
4. Securing WebLogic Server 12c,
5. Why do I need an Authenticator when I have an Identity Asserter?, Oracle A-team http://www.ateam-oracle.com/why-do-i-need-an-authenticator-when-i-have-an-identity-asserter/
6. Do You Need to Develop a Custom Authentication Provider?http://docs.oracle.com/cd/E21764_01/web.1111/e13718/atn.htm#DEVSP220
7. Weblogic Identity Asserter and Athorization Provider in one! http://darylwiest.blogspot.com.es/2015/02/weblogic-identity-asserter-and.html
Extends the mbean to an authenticator
8. Passing Tokens for Perimeter Authentication http://docs.oracle.com/cd/E21764_01/web.1111/e13718/ia.htm#DEVSP254
9. SiteMinder WebLogic Security Provider Mock, https://gibaholms.wordpress.com/2015/01/21/siteminder-weblogic-security-provider-mock/
10. Mock Weblogic Login module - Identity Asserter and Authenticator,  http://danielveselka.blogspot.com.es/2012/04/mock-weblogic-login-module-identity.html

DeadLock Detector

DeadLock Detector

Two utilities to detect a deadlock inside a JVM.

See https://github.com/carlgira/soa-utils/tree/master/deadlock

 

 deadlock-detection

A remote deadlock detector. It connects to a JVM using JMX to detect if a deadlock is ocurring. It needs the host of the JVM and the JMX port.

  $ cd deadlock-detection
  $ mvn clean package
  $ java -jar target/deadlock-detection-1.0-SNAPSHOT.jar localhos:3333

deadlock-detector-service

A Weblogic Rest service to detect if is ocurring a deadlock in the server. It can deployed on Weblogic or tested with spring-boot

• Test with Spring-boot

  $ cd deadlock-detector-service
  $ mvn clean package
  $ java -jar target/deadlock-detector-service-1.0.0.war
  $ curl -X GET http://localhost:8080/deadlock

• Weblogic

  $ cd deadlock-detector-service
  $ mvn clean package
  $ Deploy to Weblogic 
  $ curl -X GET http://localhost:7001/deadlock-detector-service-1.0.0/deadlock

Thanks! 

Soa Worklist Authentication with LdapX509Asserter

Recently I had to configure two way SSL authentication to the worklist, validating the client certificate with one saved in the Ldap.

To do this is necessary to configure the LdapX509Asserter and the LdapAuthenticator. I put all the instructions I had to follow to complete this configuration.

Create PKS12

Have or create the bundle with the certificates necessaries in pks12 format (also called pfx or just p12) to add to Weblogic

You need to add to the bundle the server certificate and the private key.

     openssl pkcs12 -name alias -export -in mycert.crt -inkey mykey.key > server.p12

Create KeyStores

Weblogic saves all the certificates in two keystores

• TrusKeyStore: Save here all the CAs
• IdentityKeyStore: Save here all the certificates and private keys

Create empty TrustKeyStore

     keytool -genkey -alias TrustAlias -keyalg RSA -keystore TrustKeyStore.jks

Import the certificates to the keystores

Importar CA a TrustKeyStore

     keytool -importcert -file my-ca-file.crt-keystore TrustKeyStore.jks -storepass TrustKeyStorePassPhrase

Import pk12 bundle to IdentityStore

     keytool -importkeystore -srckeystore server.p12 -srcstoretype pkcs12  -srcalias alias -destkeystore IdentityKeyStore.jks  -deststoretype jks -deststorepass password -destalias alias

See [2] for Common SSL commands. 

Weblogic Configuration

Transport

Follow instructions in [1] in "Transport" section.

1. Enable "SSL Listener Port" in server General tab.
2. Configure the identity and trust keystores. Sever -> Configuration -> KeyStores. Select "Custom Identity and Java Standard Trust", configure all values.
3. Specify the certificate to use for weblogic using the alias of the certificate used in the IdentityKeyStore. Go to Server -> Configuration -> SSL, configure the alias and the password of IdentityKeyStore
4. Configure Weblogic to "Two way Client Cer Behavior" Sever -> Configuration -> SSL -> Advanced,  choose "Client Cert Requested and Enforced"

Authentication

Follow instruction to configure LdapX509Asserter [3].

• Configure the ldap connection data
• User name attribute in LDAP 
• User filter attributes, the fields to match between the certificate and the LDAP entry. (check the certificate metadata and your ldap

Follow instruction to configure LdapAuthenticator [4]

• Configure the ldap connection
• Configure the user section
• Configure the group section

(Read this article [1],  it is very helpful to understand about authenticators and asserters)

The configuration of the asserter and the authenticator is not hard You need to know the data to connect to the LDAP and the information to identify the user

About those instructions there are couple of important things

1. It's important to know that the IdentityAsserters needs to work with the Authenticators. So it's necessary to configure first the LdapX509Asserter and next the LdapAuthenticator.
2. Make sure to "reorder" the asserters and authenticator in your realm. Put them in the order you want they be executed. (Security Realms -> your realm -> Providers -> Reorder)
3. If you have more than one Authenticator configure the flag "Control Flag" to "SUFFICENT" in "all" the authenticators. 
4. In the LdapX509Authenticator there is a field to configure where is the certificate. It could be "usercertificate" or "usercertificate;binary". I got a problem because my ldap browser shows me the field as "usercertificate;binary" but after testing i had to change it to "usercertificate".

Configure Custom Authenticator

Besides the configuration in the console of the LdapAuthenticator, you have change the property "idstore.type"  in the jps-config.xml.

You can change it manually in jps-config.xml [5], or using the Enterprise Manager going to,  Farm -> Weblogic Domain, right click on your domain, Security -> Security Provider Configuration. In the section "Identity Store Provider" click on "Configure" add the property "idstore.type" according with your provider

The possible values are OID ,  OVD , IPLANET , ACTIVE_DIRECTORY , EDIRECTORY , OPEN_LDAP  [6]

Update Https port in Human Task

For any already deployed workflow task detail applications, change the workflow task display URL to use the correct protocol and port number.

Go to the enterprise manager and open the afected composite. In the "Component Metrics" section click over the human task, "administration" tab, and update the https port [9]

Debug

There two things you could do to debug.

1. Activate the logger of the Authenticators. Go to Server -> Debug. Activate the logger  weblogic->security->atn->DebugSecurityAtn. (This is very useful) [7]
2. Use the next flags  -Dssl.debug=true -Dweblogic.StdoutDebugEnabled=true -Dweblogic.security.SSL.verbose=true   [8]

Test

To test that your LdapX509Asserter is working install the client certificates on your browser and open the worklist on the ssl port.

The browser it's going to show you the list of certificates that you could use to authenticate. If the certificate validates against the server and is equals to the one in the ldap you are inside the worklist.

Thanks!

References

        1. Two way SSL Configuration,   https://www.captechconsulting.com/blogs/two---way-ssl-configuration-for-web-applications

        2. Common SSL commands https://www.sslshopper.com/article-most-common-java-keytool-keystore-commands.html

       3. Configuring an LDAP X509 Identity Assertion Provider https://docs.oracle.com/cd/E13222_01/wls/docs81/secmanage/providers.html#1197612

4. Configure LdapAuthenticator https://docs.oracle.com/middleware/1213/wls/SECMG/ldap_atn.htm#SECMG175

5. Configure custom authenticators http://docs.oracle.com/cd/E28280_01/core.1111/e10043/idstoreadm.htm#JISEC9738

6. Supported LDAPhttp://docs.tpu.ru/docs/oracle/en/fmw/11.1.1.6.0/core.1111/e10043/devauthn.htm

7. Troubleshooting with authenticators https://docs.oracle.com/cd/E24628_01/doc.121/e36415/sec_troubleshoot.htm#EMSEC12981

8. SSL TroubleShooting and Debugging https://blogs.oracle.com/WebLogicServer/entry/ssl_troubleshooting_and_debugg 
9. Managing the URI of the Human Task. http://docs.oracle.com/cd/E23943_01/admin.1111/e10226/hwf_mang.htm#SOAAG3757