I am really excited about this new offering from our business unit – the official press release went out today and I could not resist blogging about it even though I am getting a bit salesy here! As Dennis said in the press release – we have been providing remote WebSphere administration and development services for various client installations that included WebSphere Application Server Network Deployment, WebSphere Portal, and IBM Web Content Management. These services ranged from complete outsourcing of all WebSphere administration and maintenance needs, to on-demand consulting and mentoring. So, in essence, we have now codified these services that we have been offering for years, into pre-packaged offerings that make it simpler for prospects and customers to understand the value proposition of Xtivia’s remote WebSphere services.

An excerpt from the news release -

Virtual-WebSphereAdmin provides a cost-effective solution for organizations seeking to optimize the availability and performance of their critical WebSphere-based business information systems. This is accomplished through a comprehensive remote WebSphere Administration service designed specifically to meet the WebSphere administration, maintenance, management and consulting needs of our clients. Xtivia’s Virtual-WebSphereAdmin service combines a sophisticated suite of monitoring and reporting tools with expert consulting services to deliver complete WebSphere performance management.

You can find more details about this service at http://www.xtivia.com/remote-services/virtual-websphereadmin or you can go directly to http://www.xtivia.com/request-information to get in touch with the Xtivia sales team about this service. I am confident that you will find it worth your time to do so!

Yes, I am back from the dead! It has been a long time since I blogged – just been very busy and blogging fell by the wayside. Recently one of the developers on my team ran into an issue with a custom theme when he applied the 6.0.1.5 fixpack on a WebSphere Portal 6.0.x.x server. The theme was exhibiting some very odd behavior – HTML snippets in the output were getting randomly misplaced – i.e. the HTML output was getting garbled with some HTML incorrectly appearing before other HTML, and the theme was having visual issues. And the odd thing was that only one of the custom themes was exhibiting the issue while other themes seemed to work well, making him believe that there was something wrong with the failing theme. He had a PMR open with IBM support for 2+ weeks but that had got him nowhere with getting the issue resolved. So I finally got involved in helping him out  and in a few minutes found this IBM technote that appeared to be related to the issue at hand. As per the technote the resolution was to simply set the buffer to none in the pages declaration. In the theme default.jsp file, we changed the page declaration to the following:

<%@ page buffer=”none” %>

That was all it took, but I sure do wish that support could actually prove to be useful once in a while and use their own tools (technotes) to resolve PMRs quickly. Of course, given that the technote was a public one, the WebSphere Portal community ought to be able to use the IBM Support Site effectively too.

I needed to implement Single Sign-On between IBM WebSphere Portal and HP Operations Dashboard (HPOD) without using a SSO product, and figured that we could do that using the LTPA token generated by WPE on login to the Portal. For LTPA token based SSO to work, we need to be able to decode the LTPA token on the HPOD front – HPOD is based on Jetspeed – in other words, we are looking at implementing SSO between WebSphere and Jetspeed. I was just getting ready to look up some info that I have from Jerry Cuomo on the LTPA token format, when I tried a quick Google search and found an even better answer. I stumbled upon a blog entry and functional code for LTPA token decoding at http://blog.offbytwo.com/2007/08/21/working-with-lightweight-third-party-authentication-ltpa/. I downloaded the code, exported the LTPA keys from a test WPE server, copied the 3DESKey and our LTPA encryption password into LtpaUtils, logged in to the WPE test server, determined the LTPA token cookie value for test purposes, and was able to decrypt it just fine using LtpaUtils. And thanks to Cosmin, all of this took about 20 minutes!

Recently we wanted to duplicate an existing Quickr test server from one machine to another without going through the entire process of installing Quickr, configuring security, configuring an external web server, installing a fixpack and efixes, installing our custom applications, and preserving all the test Quickr teamplaces and content. We got this to work pretty cleanly by restoring a file system backup of Quickr and a backup of the Quickr DB2 database. We did this for one of our test servers, but ran into an exception when we tried to duplicate another test server.

[7/14/08 10:29:36:953 CDT] 0000003b JCRCFLLoggerI E com.ibm.icm.ts.tss.JCRCFLLoggerImpl com.ibm.icm.ts.tss.ls.LibraryServerImpl.removeExpiredTableClaims [java.lang.ThreadGroup[name=icmciWorkManager: icmjcrear,maxpri=10]]: Error creating new search result table: {0}. com.ibm.db2.jcc.a.SqlException: DB2 SQL error: SQLCODE: -601, SQLSTATE: 42710, SQLERRMC: JCR.TSSTBL_1;TABLE

            at com.ibm.db2.jcc.a.lg.e(lg.java:1600)

            at com.ibm.db2.jcc.a.lg.b(lg.java:1164)

            at com.ibm.db2.jcc.b.gb.h(gb.java:217)

            at com.ibm.db2.jcc.b.gb.b(gb.java:46)

            at com.ibm.db2.jcc.b.w.b(w.java:40)

            at com.ibm.db2.jcc.b.vb.e(vb.java:118)

            at com.ibm.db2.jcc.a.lg.m(lg.java:1159)

            at com.ibm.db2.jcc.a.lg.a(lg.java:1874)

            at com.ibm.db2.jcc.a.lg.c(lg.java:520)

            at com.ibm.db2.jcc.a.lg.executeUpdate(lg.java:504)

            at com.ibm.ws.rsadapter.jdbc.WSJdbcStatement.pmiExecuteUpdate(WSJdbcStatement.java:1235)

            at com.ibm.ws.rsadapter.jdbc.WSJdbcStatement.executeUpdate(WSJdbcStatement.java:738)

            at com.ibm.icm.ts.tss.ls.BaseDBImpl.executeUpdate(BaseDBImpl.java:248)

            at com.ibm.icm.ts.tss.ls.BaseDBImpl.createTable(BaseDBImpl.java:306)

            at com.ibm.icm.ts.tss.ls.LibraryServerImpl.removeExpiredTableClaims(LibraryServerImpl.java:450)

            at com.ibm.icm.ts.tss.ls.LibraryServerImpl.state(LibraryServerImpl.java:53)

            at com.ibm.icm.ts.tss.app.Janitor.cleanupLibraryServer(Janitor.java:43)

            at com.ibm.icm.ts.tss.app.JanitorService.cleanupLibraryServer(JanitorService.java:66)

            at com.ibm.icm.ts.tss.app.JanitorService.access$000(JanitorService.java:22)

            at com.ibm.icm.ts.tss.app.JanitorService$Notifiee.onEvent(JanitorService.java:110)

            at com.ibm.icm.ts.act.ActivityImpl$ManagerImpl$ActivityRunner.runActivity(ActivityImpl.java:333)

            at com.ibm.icm.ts.act.ActivityImpl$ManagerImpl$ActivityRunner.run(ActivityImpl.java:303)

            at com.ibm.hrl.workmanager.RunnableWrapper.run(RunnableWrapper.java:44)

            at com.ibm.hrl.workmanager.impl.was.WasWork.run(WasWork.java:75)

            at com.ibm.ws.asynchbeans.J2EEContext$RunProxy.run(J2EEContext.java:258)

            at java.security.AccessController.doPrivileged1(Native Method)

            at java.security.AccessController.doPrivileged(AccessController.java(Compiled Code))

            at javax.security.auth.Subject.doAs(Subject.java:477)

            at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.java:118)

            at com.ibm.ws.asynchbeans.J2EEContext$DoAsProxy.run(J2EEContext.java:325)

            at java.security.AccessController.doPrivileged1(Native Method)

            at java.security.AccessController.doPrivileged(AccessController.java:351)

            at com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:709)

            at com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:218)

            at com.ibm.ws.asynchbeans.ABWorkItemImpl.run(ABWorkItemImpl.java:154)

            at java.lang.Thread.run(Thread.java:570)

[7/14/08 10:29:36:969 CDT] 0000003b JCRCFLLoggerI E com.ibm.icm.ts.tss.JCRCFLLoggerImpl com.ibm.icm.ts.tss.app.Janitor.cleanupLibraryServer [java.lang.ThreadGroup[name=icmciWorkManager: icmjcrear,maxpri=10]]: Error getting the library server. com.ibm.icm.ts.tss.ls.DatabaseException: com.ibm.db2.jcc.a.SqlException: DB2 SQL error: SQLCODE: -601, SQLSTATE: 42710, SQLERRMC: JCR.TSSTBL_1;TABLE

            at com.ibm.icm.ts.tss.ls.LibraryServerImpl.removeExpiredTableClaims(LibraryServerImpl.java:455)

            at com.ibm.icm.ts.tss.ls.LibraryServerImpl.state(LibraryServerImpl.java:53)

            at com.ibm.icm.ts.tss.app.Janitor.cleanupLibraryServer(Janitor.java:43)

            at com.ibm.icm.ts.tss.app.JanitorService.cleanupLibraryServer(JanitorService.java:66)

            at com.ibm.icm.ts.tss.app.JanitorService.access$000(JanitorService.java:22)

            at com.ibm.icm.ts.tss.app.JanitorService$Notifiee.onEvent(JanitorService.java:110)

            at com.ibm.icm.ts.act.ActivityImpl$ManagerImpl$ActivityRunner.runActivity(ActivityImpl.java:333)

            at com.ibm.icm.ts.act.ActivityImpl$ManagerImpl$ActivityRunner.run(ActivityImpl.java:303)

            at com.ibm.hrl.workmanager.RunnableWrapper.run(RunnableWrapper.java:44)

            at com.ibm.hrl.workmanager.impl.was.WasWork.run(WasWork.java:75)

            at com.ibm.ws.asynchbeans.J2EEContext$RunProxy.run(J2EEContext.java:258)

            at java.security.AccessController.doPrivileged1(Native Method)

            at java.security.AccessController.doPrivileged(AccessController.java(Compiled Code))

            at javax.security.auth.Subject.doAs(Subject.java:477)

            at com.ibm.websphere.security.auth.WSSubject.doAs(WSSubject.java:118)

            at com.ibm.ws.asynchbeans.J2EEContext$DoAsProxy.run(J2EEContext.java:325)

            at java.security.AccessController.doPrivileged1(Native Method)

            at java.security.AccessController.doPrivileged(AccessController.java:351)

            at com.ibm.ws.asynchbeans.J2EEContext.run(J2EEContext.java:709)

            at com.ibm.ws.asynchbeans.WorkWithExecutionContextImpl.go(WorkWithExecutionContextImpl.java:218)

            at com.ibm.ws.asynchbeans.ABWorkItemImpl.run(ABWorkItemImpl.java:154)

            at java.lang.Thread.run(Thread.java:570)

Caused by: com.ibm.db2.jcc.a.SqlException: DB2 SQL error: SQLCODE: -601, SQLSTATE: 42710, SQLERRMC: JCR.TSSTBL_1;TABLE

            at com.ibm.db2.jcc.a.lg.e(lg.java:1600)

            at com.ibm.db2.jcc.a.lg.b(lg.java:1164)

            at com.ibm.db2.jcc.b.gb.h(gb.java:217)

            at com.ibm.db2.jcc.b.gb.b(gb.java:46)

            at com.ibm.db2.jcc.b.w.b(w.java:40)

            at com.ibm.db2.jcc.b.vb.e(vb.java:118)

            at com.ibm.db2.jcc.a.lg.m(lg.java:1159)

            at com.ibm.db2.jcc.a.lg.a(lg.java:1874)

            at com.ibm.db2.jcc.a.lg.c(lg.java:520)

            at com.ibm.db2.jcc.a.lg.executeUpdate(lg.java:504)

            at com.ibm.ws.rsadapter.jdbc.WSJdbcStatement.pmiExecuteUpdate(WSJdbcStatement.java:1235)

            at com.ibm.ws.rsadapter.jdbc.WSJdbcStatement.executeUpdate(WSJdbcStatement.java:738)

            at com.ibm.icm.ts.tss.ls.BaseDBImpl.executeUpdate(BaseDBImpl.java:248)

            at com.ibm.icm.ts.tss.ls.BaseDBImpl.createTable(BaseDBImpl.java:306)

            at com.ibm.icm.ts.tss.ls.LibraryServerImpl.removeExpiredTableClaims(LibraryServerImpl.java:450)

            ... 21 more

From the stacktrace, it was clear that the issue was related to Quickr trying to create a table named TSSTBL_1 that already existed. From one of the IBM performance tuning notes, I knew that on the very first search, the system creates twenty database tables named TSSTBL_1, TSSTBL_2 … TSSTBL_20 in the JCR database for search information. That note also talked about the fact that the system can dynamically generate more TSSTBLs. From our earlier investigations of JCR replication, I knew about the table named ICMSTJCRTSTABLES that tracks what TSSTBLs exist. On looking into the Quickr DB2 database I could see that there were the 20 TSSTBLs and there were 20 rows in ICMSTJCRTSTABLES – one for each of them. Given that Quickr (or WebSphere Portal) automatically creates the TSSTBLs on the first search, I figured that if I deleted the TSSTBLs and the 20 rows in ICMSTJCRTSTABLES, I could get Portal to recreate the tables and things ought to work right without this extraneous exception cluttering up the log. Well, that is what I did (drop the TSSTBL tables and deleting all the rows from ICMSTJCRTSTABLES using the DB2 command editor – first time that I ever used that), restarted Portal (actually Quickr), performed a search, and sure enough the TSSTBL tables got recreated and the exception was gone. I was pretty pleased with myself for figuring out this solution in about a hour – now did I ever claim to be modest?!

A short and sweet entry to document the quick and easy resolution for the situation where you encounter the following error starting up the WebSphere Portal service on Windows -

The IBM WebSphere Application Server V6 – WebspherePortal service terminated with service-specific error 134221464 (0×8000E98).

This is a problem where the service fails to start up, and the failure occurs real quickly. Essentially, the WebSphere Portal java process is not even kicked off and you do not find any logging in profiles\${profile_name}\logs\WebSphere_Portal\startServer.log for the WebSphere Portal server. I have encountered this a few times, and usually for me, the issue has been related to an ungraceful shutdown of WebSphere Portal. I have restarted my Portal VMWare VM unexpectedly and the WebSphere_Portal.pid file is left in the profiles\${profile_name}\logs\WebSphere_Portal. The resolution for this issue is to simply delete the pid file and to restart the service.

This is a very trivial entry for most people in the know, but not so trivial for others. If you are configuring WebSphere Portal to use either Active Directory or Active Directory Application Mode (ADAM) as the LDAP user registry, then it is critical to set up the user search filter correctly or else you are going to run into issues with users not being able to authenticate. We have a client for whom we are using ADAM wherein ADAM is used for the Portal LDAP and it syncs up with Active Directory. Well, my portal administrator who is working on that project ran into a problem with Portal users not being able to authenticate successfully.

This issue occurs for all users except for the initial test user who happened to have his cn set to the same value as his sAMAccountName. I took a look into the issue and very quickly it was apparent where the problem lay. Essentially, the LDAPUserFilter in wpconfig.properties was set to (&(cn=%v)(objectclass=user)) as is recommended for ADAM in the Info Center. However, what this implies is that users must specify their “common name” instead of their sAMAccountName to login – obviously not a good scenario. Instead, the LDAPUserFilter in wpconfig.properties should be set to (&(|(cn=%v)(samAccountName=%v))(objectclass=user)) to allow users to login using their sAMAccountName or their common name. This is correctly suggested as the user search filter for Active Directory in the InfoCenter.

Today one of the portal administrators in my team was trying to reconfigure the LDAP server being used by an existing WebSphere Portal v6 install. The specific reconfiguration that he was doing was extremely simple – he was replacing one LDAP server with another identically configured LDAP server – so the only change was the name of the LDAP server. We were able to perform this change without disabling/re-enabling security in WebSphere Portal.

You end up changing two files to change the LDAP server name -

• security.xml: This file reconfigures the LDAP server being used by WebSphere Application Server. You can find this file in ${WP_PROFILE_ROOT}\config\cells\<cell_name>. You change the LDAP server name (highlighted in red) in the following lines -

<userRegistries xmi:type=”security:LDAPUserRegistry” xmi:id=”LDAPUserRegistry_1″ serverId=”wasadmin_dn” serverPassword=”{xor}wasadmin_pwd” realm=”ldap_server_name:389″ limit=”0″ ignoreCase=”true” type=”CUSTOM” sslEnabled=”true” sslConfig=”test/DefaultSSLSettings” baseDN=”dc=com” bindDN=”wpsbind_dn” bindPassword=”{xor}wpsbind_pwd” searchTimeout=”120″ reuseConnection=”true”>
<searchFilter xmi:id=”LDAPSearchFilter_1″ userFilter=”(&(uid=%v)(objectclass=inetOrgPerson))” groupFilter=”(&(cn=%v)(objectclass=groupOfUniqueNames))” userIdMap=”*:uid” groupIdMap=”*:cn” groupMemberIdMap=”groupOfUniqueNames:uniqueMember” certificateMapMode=”EXACT_DN” certificateFilter=”"/>
<hosts xmi:id=”EndPoint_1173225214609″ host=”ldap_server_name” port=”389″/>
</userRegistries>

• wmm.xml: This file reconfigures the LDAP server being used by WebSphere Portal WMM component. You can find this file in ${WP_SERVER_ROOT}\wmm. You change the LDAP server name (highlighted in red) in the following XML snippet -

<ldapRepository name=”wmmLDAP”
UUID=”LDAP1″
adapterClassName=”com.ibm.ws.wmm.ldap.ibmdir.IBMDirectoryAdapterImpl”
supportDynamicAttributes=”false”
configurationFile=”wmmLDAPServerAttributes.xml”
wmmGenerateExtId=”false”
supportGetPersonByAccountName=”true”
profileRepositoryForGroups=”LDAP1″
supportTransactions=”false”
adminId=”admin_dn”
adminPassword=”admin_pwd”
ldapHost=”ldap_server_name“
ldapPort=”389″
ldapTimeOut=”6000″
ldapAuthentication=”SIMPLE”
ldapType=”0″
sslEnabled=”true”
sslTrustStore=”C:\WebSphere\AppServer\etc\DummyServerTrustFile.jks”
dirContextsMaxSize=”20″
dirContextsMinSize=”5″
dirContextTimeToLive=”-1″
cacheGroups=”false”
groupsCacheTimeOut=”600″
cacheAttributes=”true”
attributesCacheSize=”2000″
attributesCacheTimeOut=”600″
cacheNames=”true”
namesCacheSize=”2000″
namesCacheTimeOut=”600″>

Once you have changed the LDAP server name in these two files to match the new server name, restart WebSphere Portal and you should be set to go.

Open up SystemOut.log in ${WP_SERVER_ROOT}\log and look for the following line during server startup to verify that you are using the new LDAP server -

[6/18/08 12:10:34:690 CDT] 0000000a LdapRegistryI A SECJ0419I: The user registry is currently connected to the LDAP server ldap://wpsldap:389.

In our case, the LDAP server name is “wpsldap”.

My Tip: In our test environments, we usually follow the practice of using virtual host names for the LDAP and database server. Rather than configuring the actual host names of the LDAP and database server being used by a given WebSphere Portal server, we use hosts file entries on Windows that configure wpsldap and wpsdb to point to the IP addresses of the relevant LDAP and database servers. Whenever we need to reconfigure a test WP server to use a different LDAP/database server, all we need to do is change the IP address in the hosts file to point to the new server. Simple, huh! And not exactly rocket science!

Recently our team needed to deploy the Portal Document Manager (PDM) portlet on a public WebSphere Portal page for guest user access. It turned out that this was not as trivial as expected if you don’t know what actions are required and there is nothing on IBM’s site documenting the steps involved in getting the document manager portlet to work for guest users. I figured I would capture the key steps here in case you need to do the same and are having a hard time with this task!

1. Enable public sessions in WebSphere Portal by adding public.session=true to the Navigator Service properties file. Refer to the WP Infocenter for help with this task.
2. Give anonymous user at least “User” level access to the “document manager” portlet.
3. Give anonymous user at least “User” level access to the “document library” being displayed in the “document manager” portlet.
4. Add “Everyone” to the “Authenticated” role of the “document manager” web application at the WebSphere Application Server level using the WAS admin console.

Disclaimer: I owe Dinesh on my team the credit for figuring out the last and most critical step in ensuring that the PDM portlet worked for guest or anonymous users. Kudos, Dinesh!

This blog entry is straight plagiarism! I got this from Keith (a co-worker) and thought that it was pretty cool and decided to share his work. Keith has been doing some cool stuff for an Xtivia client and some of this work has been around some serious theme customizations. One of those theme customizations that he had to figure out was how to add a persistent link to the theme which will add a new page at the currently selected level of navigation in WPE. This is useful for many situations including where you have a tab based navigation, you can have a “New page” tab where users can simply specify a page name and add a new tab at the same level as the other tabs. It goes like this:

<c:forEach var="i" begin="${themePolicy.topNavigationStartLevel}" end="${themePolicy.topNavigationStopLevel}" step="1">

<portal-navigation:navigation startLevel="${i}" stopLevel="${i}"><%-- New page --%>

<%

NavigationNode currentNode = (NavigationNode)wpsSelectionModel.getSelectedNode();

NavigationNode parentNode = (NavigationNode)wpsNavModel.getParent(currentNode);

String parentIDString = com.ibm.wps.portlets.admin.AdminPortletUtils.getStringObjectID(parentNode.getContentNode().getObjectID());

%>

<portal-internal:adminlinkinfo name="PROPERTIES_PORTLET">

<portal-navigation:urlGeneration contentNode="<%=wpsContentNode%>"

layoutNode='<%= wpsCompositionNode %>'

portletWindowState="Solo"

accessControlCheck="CreatePage"

themeTemplate="">

<portal-navigation:urlParam name="init" value="init" />

<portal-navigation:urlParam name="OCN" value="<%= parentIDString %>" />

<portal-navigation:urlParam name="WCN" value="<%= parentIDString %>"/>

<portal-navigation:urlParam name="PAT" value="PTQ" />

<portal-navigation:urlParam name="PNT" value="PNTC" />

<portal-navigation:urlParam name="psh" value="push" />

<a href="<% wpsURL.write(out); %>">New Page</a>

</portal-navigation:urlGeneration>

</portal-internal:adminlinkinfo>

</portal-navigation:navigation>

</c:forEach>

Direct Keith quote: It could be cleaned up, but you get the idea. The relevant points are that you need to pass the parent’s ObjectID String into the OCN and WCN url parameters for the urlGeneration tag, and you need to use the AdminPortletUtils.getStringObjectID(ObjectID id) in order to transform an ObjectID into a String.

Hoping that my plagiarism comes handy for somebody!

Yes, I am back with a blog entry after quite a gap here! I have been knocked out sick for 5 weeks now and I am sick of being sick! Better now and hoping to be back at a 100% soon. Enough excuses – onto the subject of this blog entry. We use Log4j’s DailyRollingFileAppender in a WebSphere Portal environment to generate daily log files for various custom portlets that we have deployed. This worked just fine on some WPE v6 test servers with multiple portlets logging to the same log file, and the daily rollover occurring every midnight. However, on some other WPE v6 test servers running the exact same applications, while logging did work the daily rollover did not. Instead of us having multiple log files with date timestamps in their names, we would only have today’s log file – the previous log files would simply disappear! This issue results from the fact that the DailyRollingFileAppender (DRFA) tries to rename the log file during rollover. However, when the rename fails, it ends up overwriting the log file, hence losing information. On checking the application server’s SystemErr.log, we found an error each midnight similar to -

log4j:ERROR Failed to rename [logfilename] to logfilename.2008.01.11].

This appears to be a fairly common issue with log4j 1.2.x in Java application servers running on Windows. While there is no good resolution for this issue there is a workaround that works pretty well.

We ended up getting the DatedFileAppender which is a log4j extension that is publicly available at http://minaret.biz/tips/datedFileAppender.html. DatedFileAppender avoids the need for renaming the log file by simply creating the log file initially with the date in the file name – this is a simple workaround for the problem that we were experiencing. Switching from the DailyRollingFileAppender to DatedFileAppender was fairly seamless.  In addition to installing the jar in WebSphere Portal, we had to switch a few lines of our log4j configuration. We changed the following lines -

log4j.appender.logFileAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.logFileAppender.DatePattern=’.'yyyy-MM-dd
log4j.appender.logFileAppender.File=MyApps.log

to

log4j.appender.logFileAppender=biz.minaret.log4j.DatedFileAppender
log4j.appender.logFileAppender.Directory=.
log4j.appender.logFileAppender.Prefix=MyApps_
log4j.appender.logFileAppender.Suffix=.log
log4j.appender.sametimeSyncAppender.Period=month # this line is only needed if the period is not a day