With a little side of applesauce...

Sunday, January 28, 2007

Coldfusion7 - Configuration information is not shared between the clustered instances

Our current configuration consists of a load-balanced web farm, and a two-machine coldfusion cluster. For example:

Apache2 - 192.168.1.30, 192.168.1.31, 192.168.1.32
Coldfusion7 - 192.168.1.40, 192.168.1.41

Configuration information is not shared between the clustered instances:

This was the first problem that we ran into. For some reason, configuration information for the coldfusion instance is not shared between the nodes. For example, DSN “x” created on coldfusion node1 was not replicated to coldfusion node2, meaning that calls to node2 for DSN “x” would return a DSN not found error.

Since we are using a central storage area for our web roots, we were able to push our jrun4 directories into a central location:

user@mymachine:/remotestorage/jrun4$ ls /remotestorage/jrun4
cf_common_files lib cfinstance1 cfinstance2

As you can see, we have moved our jrun4 installation directory onto our remotestorage:

user@mymachine:/remotestorage/jrun4$ ls cfinstance1
bin hotfix logs pointbase uninstall
C jre Macromedia_ColdFusion_MX_7.0.2_InstallLog.log PointBaseDebug.log updater6-backup
cfmx_updater_installer_debug.txt jre_installerbackup Macromedia_ColdFusion_MX_7_InstallLog.log Readme.htm verity
connectors lib Macromedia_ColdFusion_MX_7_Updater_InstallLog.log relnotes.htm
docs license.txt Macromedia_JRun_4_Updater_6_InstallLog.log servers
user@mymachine:/remotestorage/jrun4$ ls cfinstance2
bin docs license.txt Macromedia_JRun_4_Updater_6_InstallLog.log servers
C hotfix logs pointbase uninstall
cfmx_updater_installer_debug.txt jre Macromedia_ColdFusion_MX_7_InstallLog.log Readme.htm updater6-backup
connectors lib Macromedia_ColdFusion_MX_7_Updater_InstallLog.log relnotes.htm verity

As well as our jrunserver.store, so that it is accessible to all of the Apache2 machines:

user@mymachine:/remotestorage/jrun4$ ls lib/wsconfig/1/
ApacheBuildInstructions.txt build_jrun20 jrunserver.store jrunserver.store.bak mod_jrun20.so

Next, we moved all files which are shared between the instances into the /remotestorage/jrun4/cf_common_files directory. These include:

…/myinstance/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/cfx

…/myinstance/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/CustomTags

Plus, the neo-* files and password.properties from:

…/myinstance/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib/

user@mymachine:/remotestorage/jrun4$ ls cf_common_files/
cfx CustomTags lib
user@mymachine:/remotestorage/jrun4$ ls cf_common_files/lib
neo-archivedeploy.xml neo-debug.bak neo-event.xml neo-mail.bak neo-query.bak neo-runtime.xml neo-security.xml password.properties
neo-clientstore.xml neo-debug.xml neo-graphing.xml neo-mail.xml neo-query.xml neo-search.bak neo-verity.xml
neo-comobjmap.xml neo-document.bak neo_jaas.policy neo-metric.xml neo-registry.xml neo-search.xml neo-watch.xml
neo-cron.xml neo-document.xml neo-logging.xml neo-probe.xml neo-runtime.bak neo-security.bak neo-xmlrpc.xml

These are then symlinked back into the cfinstance directory trees:

user@mymachine:/remotestorage/jrun4$ ls -l myinstance1/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/
lrwxrwxrwx 1 www-data www-data 38 2006-09-05 12:55 cfx -> /remotestorage/jrun4/cf_common_files/cfx/
lrwxrwxrwx 1 www-data www-data 45 2006-09-05 12:54 CustomTags -> /remotestorage/jrun4/cf_common_files/CustomTags/

user@mymachine:/remotestorage/jrun4$ ls -l myinstance1/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib | grep neo
lrwxrwxrwx 1 www-data www-data 59 2006-09-05 13:29 neo-archivedeploy.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-archivedeploy.xml
lrwxrwxrwx 1 www-data www-data 57 2006-09-05 13:29 neo-clientstore.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-clientstore.xml
lrwxrwxrwx 1 www-data www-data 55 2006-09-05 13:29 neo-comobjmap.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-comobjmap.xml
lrwxrwxrwx 1 www-data www-data 50 2006-09-05 13:29 neo-cron.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-cron.xml
lrwxrwxrwx 1 www-data www-data 51 2006-09-05 13:29 neo-debug.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-debug.bak
lrwxrwxrwx 1 www-data www-data 51 2006-09-05 13:29 neo-debug.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-debug.xml
lrwxrwxrwx 1 www-data www-data 54 2006-09-05 13:29 neo-document.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-document.bak
lrwxrwxrwx 1 www-data www-data 54 2006-09-05 13:29 neo-document.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-document.xml
lrwxrwxrwx 1 www-data www-data 51 2006-09-05 13:29 neo-event.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-event.xml
lrwxrwxrwx 1 www-data www-data 54 2006-09-05 13:29 neo-graphing.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-graphing.xml
lrwxrwxrwx 1 www-data www-data 53 2006-12-19 07:14 neo_jaas.policy -> /remotestorage/jrun4/cf_common_files/lib/neo_jaas.policy
lrwxrwxrwx 1 www-data www-data 53 2006-09-05 13:29 neo-logging.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-logging.xml
lrwxrwxrwx 1 www-data www-data 50 2006-09-05 13:29 neo-mail.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-mail.bak
lrwxrwxrwx 1 www-data www-data 50 2006-09-05 13:29 neo-mail.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-mail.xml
lrwxrwxrwx 1 www-data www-data 52 2006-09-05 13:29 ncoldfusion 7 clustereo-metric.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-metric.xml
lrwxrwxrwx 1 www-data www-data 51 2006-09-05 13:29 neo-probe.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-probe.xml
lrwxrwxrwx 1 www-data www-data 51 2006-09-05 13:29 neo-query.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-query.bak
lrwxrwxrwx 1 www-data www-data 51 2006-09-05 13:29 neo-query.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-query.xml
lrwxrwxrwx 1 www-data www-data 54 2006-09-05 13:29 neo-registry.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-registry.xml
lrwxrwxrwx 1 www-data www-data 53 2006-09-05 13:29 neo-runtime.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-runtime.bak
lrwxrwxrwx 1 www-data www-data 53 2006-09-05 13:29 neo-runtime.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-runtime.xml
lrwxrwxrwx 1 www-data www-data 52 2006-09-05 13:29 neo-search.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-search.bak
lrwxrwxrwx 1 www-data www-data 52 2006-09-05 13:29 neo-search.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-search.xml
lrwxrwxrwx 1 www-data www-data 54 2006-09-05 13:29 neo-security.bak -> /remotestorage/jrun4/cf_common_files/lib/neo-security.bak
lrwxrwxrwx 1 www-data www-data 54 2006-09-05 13:29 neo-security.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-security.xml
lrwxrwxrwx 1 www-data www-data 52 2006-09-05 13:29 neo-verity.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-verity.xml
lrwxrwxrwx 1 www-data www-data 51 2006-09-05 13:29 neo-watch.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-watch.xml
lrwxrwxrwx 1 www-data www-data 52 2006-09-05 13:29 neo-xmlrpc.xml -> /remotestorage/jrun4/cf_common_files/lib/neo-xmlrpc.xml
user@mymachine:/remotestorage/jrun4$ ls -l myinstance1/servers/cfusion/cfusion-ear/cfusion-war/WEB-INF/cfusion/lib | grep pass
lrwxrwxrwx 1 www-data www-data 57 2006-09-05 13:29 password.properties -> /remotestorage/jrun4/cf_common_files/lib/password.properties

Once the symlinks are in place, you can restart the coldfusion server instances, and they will now share the configuration changes which take place in the coldfusion administrator. To highlight a few files:

1. neo-query.xml – contains your DSNs.
2. password.properties – contains the coldfusion administrator password

Any modified to the neo-* files will be automatically found by the coldfusion instances and they will reload the appropriate service with a log entry like:

12/19 08:28:52 Information [scheduler-1] - neo-query.xml has been modified . Reloading the service coldfusion.sql.Executive@13b3b7e

No comments: