This page last changed on Mar 03, 2010 by jlargman.
 | Clustering is vastly improved in Confluence 3.0. See the Confluence 3 Performance improvements page for more information. If you're having cluster performance problems in a version prior to Confluence 3, we suggest an upgrade! |
On this page:
Symptoms
Below is a list of potential problems with a Confluence cluster, and their likely solutions. The solutions are listed below.
Confluence cluster debugging tools
There is an umbrella issue opened for all cluster debugging tools here
It includes the tools listed below.
Multicast
The multicast address and port used by Confluence can be found on the Cluster Administration page, or in confluence.cfg.xml in the Confluence home directory.
- Multicast address generation.
Confluence uses a hashing algorithm to take the inputted name during setup and it is then turned into a multicast address stored in the config file. Thus, once the initial setup is completed, Confluence will use the address this is the reason why user can change the address if needed, without actually changing the name. Consequently the additional nodes using the same multicast address specified in the config file are able to join the cluster.
Each node has a multicast address configured in the confluence-cfg.xml file
name="confluence.cluster.address">xxx.xx.xxx.xxx</property>
A warning message is displayed when an user changes the address from the one that Confluence has generated by the hashing of the name. There is no way of eliminating the message any other way other than by returning the address to the one that matches the cluster name. Purpose of the warning message is to remind the user that the address has been changed - as it is not the hashed version any longer - consequently the node can not join the cluster just by using the name. It is also necessary to provide the correct address as well.
Mapping interface to IP address.
To ensure that the interface name is mapped correctly, the following tool can be used. It shows the mapping of the interface name to the IP address.
Debugging tools
Listed below are some debugging tools that help determine what the status of the multicast traffic is:
Tool |
Information provided |
netstat -gn |
Lists multicast groups. Does not work on Mac OS X. |
netstat -rn |
Lists system routing table. |
Multicast Test |
Coherence tool for testing multicast traffic from one node to another. |
tcpdump -i interface |
Captures network traffic on the given interface. Most useful on an interface that only receives cluster traffic. |
Add multicast route
Multicast networking requirements vary across operating systems. Some operating systems require little configuration, while some require the multicast address to be explicitly added to a network interface before Confluence can use it.
If the Multicast Test tool shows that multicast traffic can't be sent or received correctly, adding a route for multicast traffic on the correct interface will often fix the problem. The example below is for a Ubuntu Linux system:
To support multiple applications using multicast on different interfaces, you may need to specify a route specific to the Confluence multicast address.
Check firewall
Ensure your firewall allows UDP traffic on the multicast address and port used by Confluence.
Prefer IPv4
 | There's a known issue with IPv6, especially on Linux. |
The fix is to add -Djava.net.preferIPv4Stack=true to JAVA_OPTS. This tells the JVM to try binding an IPv4 address first, and resort to IPv6 only if that fails.
Note: A more radical approach is to add NETWORKING_IPV6=no to /etc/sysconfig/network, yet probably should be left for a later consideration on a production machine.
Change multicast interface
Confluence might have selected the incorrect interface for multicast traffic, which means it cannot connect to other nodes in the cluster. To override the interface used for multicast traffic after initial setup, edit confluence.cfg.xml in the Confluence home directory and add a property (or change the existing one) to select your desired network interface. For example to tell Confluence to use eth1:
<property name="confluence.cluster.interface">eth1</property>
Increase multicast TTL
The multicast time-to-live (TTL) specifies how many hops a multicast packet should be allowed to travel before it is discarded by a router. It should be set to the number of routers in between your clustered nodes: 0 if both are on the same machine, 1 if on two different machines linked by a switch or cable, 2 if on two different machines with one intermediate router, and so on.
Create a file in the Confluence home directory called tangosol-coherence-override.xml. Add the following to it, setting the TTL value appropriately (1 is the default):
<?xml version='1.0'?>
<coherence>
<cluster-config>
<multicast-listener>
<time-to-live system-property='tangosol.coherence.ttl'>1</time-to-live>
</multicast-listener>
</cluster-config>
</coherence>
Alternatively, simply start Confluence with the system property: -Dtangosol.coherence.ttl=1. Again, 1 is the default value, and you should change it to something appropriate to your network topology.
Check intermediate routers
Advanced switches and routers have the ability to understand multicast traffic, and route it appropriately. Unfortunately sometimes this functionality doesn't work correctly with the multicast management information (IGMP) published by the operating system running Confluence.
If multicast traffic is problematic, try disabling advanced multicast features on switches and routers in between the clustered nodes. These features can prevent multicast traffic being transmitted by certain operating systems.
For best results, use the simplest network topology possible for the cluster traffic between the nodes. For two nodes, that means a single network cable. For larger numbers, try using a single high-quality switch.
Advanced Tangosol configuration
If the solution to your problem involves changes to the Tangosol configuration, these changes should not be made to the Confluence configuration in confluence/WEB-INF/classes/. Instead, to ensure your configuration survives upgrades, make your changes via:
- Tangosol system properties
- creating a tangosol-coherence-override.xml file in the Confluence home directory.
Examples of making these changes are shown in the increasing the TTL section.
Didn't find a solution?
Check Related Articles from the Confluence Knowledge Base
Open JIRA Features and Bug Reports
JIRA Issues (54 issues)
|
Type |
Key |
Summary |
Assignee |
Reporter |
Priority |
Status |
Resolution |
Created |
Updated |
Due |
|
CONF-8959
|
Attachment migration does not happen when upgrading to a clustered license
|
Unassigned
|
Nicholas Ilacqua [Atlassian]
|
|
Open
|
Unresolved
|
Jul 19, 2007
|
Jul 02, 2009
|
|
|
CONF-9297
|
Confluence should be able to automatically recover from cluster panics
|
Unassigned
|
Gary Weaver
|
|
Open
|
Unresolved
|
Aug 27, 2007
|
Mar 25, 2009
|
|
|
CONF-12287
|
Coherence cache fails while retrieving profile picture metadata (dashboard or view page shows UnexpectedRollbackException)
|
Daniel Kjellin [Atlassian]
|
Matt Ryall [Atlassian]
|
|
Open
|
Unresolved
|
Jul 01, 2008
|
Apr 20, 2010
|
|
|
CONF-12689
|
Support Confluence cluster upgrades without an outage
|
Unassigned
|
Igor Minar
|
|
Open
|
Unresolved
|
Aug 06, 2008
|
May 04, 2010
|
|
|
CONF-14120
|
Hibernates UpdateTimestampsCache doesn't handle concurrent writes
|
Unassigned
|
Chris Kiehl [Atlassian]
|
|
Open
|
Unresolved
|
Jan 05, 2009
|
May 05, 2009
|
|
|
CONF-16794
|
Document new cluster distributions
|
Giles Gaskell [Atlassian]
|
Jeremy Largman [Atlassian]
|
|
Open
|
Unresolved
|
Sep 04, 2009
|
May 12, 2010
|
|
|
CONF-8716
|
Determine index mismatch in cluster and warn on cluster info page
|
Unassigned
|
Matt Ryall [Atlassian]
|
|
Open
|
Unresolved
|
Jun 17, 2007
|
Oct 29, 2007
|
|
|
CONF-9040
|
Authenticator (subclass of DefaultAuthenticator) can be called twice at almost exactly same time by 2 or more clustered servers
|
Unassigned
|
Gary Weaver
|
|
Open
|
Unresolved
|
Jul 30, 2007
|
Nov 04, 2007
|
|
|
CONF-9324
|
Lots of ObjectDeletedException's during cluster builds
|
Unassigned
|
Matthew Jensen [Atlassian]
|
|
Open
|
Unresolved
|
Aug 28, 2007
|
May 12, 2010
|
|
|
CONF-9594
|
ConditionalPropertySet's cannot be cached breaking cluster installations that delegate user management to JIRA
|
Unassigned
|
Dave Loeng [Atlassian]
|
|
Open
|
Unresolved
|
Sep 28, 2007
|
Jul 02, 2009
|
|
|
CONF-9813
|
Disable attachments migration to Filesystem in Cluster
|
Unassigned
|
Gurleen Anand [Atlassian]
|
|
Open
|
Unresolved
|
Oct 24, 2007
|
Oct 25, 2007
|
|
|
CONF-10054
|
OSUser provider does not generate IDs correctly in clustered environment
|
Unassigned
|
Christopher Owen [Atlassian]
|
|
Open
|
Unresolved
|
Nov 26, 2007
|
Dec 01, 2008
|
|
|
CONF-10323
|
Coherence Lock being held when it appears no thread should have the lock. Causes ConcurrentModificationException
|
Unassigned
|
Paul Curren [Atlassian]
|
|
Open
|
Unresolved
|
Dec 26, 2007
|
Jan 14, 2010
|
|
|
CONF-10325
|
Viewing the members of a group in a clustered environment works only on one node and not the other.
|
Unassigned
|
Partha Kamal [Atlassian]
|
|
Open
|
Unresolved
|
Dec 27, 2007
|
Jul 02, 2009
|
|
|
CONF-10868
|
Node that can not join cluster due to license restriction causes cluster panic
|
Unassigned
|
Ivan Benko [Atlassian]
|
|
Open
|
Unresolved
|
Feb 29, 2008
|
Sep 03, 2008
|
|
|
CONF-10980
|
Cluster debugging/troubleshooting tools
|
Unassigned
|
Ivan Benko [Atlassian]
|
|
Open
|
Unresolved
|
Mar 06, 2008
|
Mar 06, 2008
|
|
|
CONF-12345
|
Park issue :)
|
Unassigned
|
Dave Loeng [Atlassian]
|
|
Open
|
Unresolved
|
Jul 07, 2008
|
Jan 21, 2009
|
|
|
CONF-12486
|
ClassNotFoundException logged on cluster node startup
|
Unassigned
|
Anatoli Kazatchkov [Atlassian]
|
|
Open
|
Unresolved
|
Jul 17, 2008
|
Aug 25, 2009
|
|
|
CONF-12614
|
Intermittent ConcurrentModificationException in cluster
|
Unassigned
|
Anatoli Kazatchkov [Atlassian]
|
|
Open
|
Unresolved
|
Jul 29, 2008
|
Mar 31, 2009
|
|
|
CONF-13421
|
Layout customisations are not propagated to other cluster nodes
|
Unassigned
|
Matt Ryall [Atlassian]
|
|
Open
|
Unresolved
|
Oct 16, 2008
|
Dec 09, 2008
|
|
|
CONF-14045
|
Cluster mysql tests corrupts the database
|
Unassigned
|
Brian Nguyen [Atlassian]
|
|
Open
|
Unresolved
|
Dec 21, 2008
|
Jan 08, 2009
|
|
|
CONF-14076
|
SchedulerException when running cluster builds
|
Unassigned
|
Brian Nguyen [Atlassian]
|
|
Open
|
Unresolved
|
Dec 29, 2008
|
Jan 01, 2009
|
|
|
CONF-14657
|
Retrieving the global settings in a clustered environment causes a lot of contention
|
Unassigned
|
Chris Kiehl [Atlassian]
|
|
Open
|
Unresolved
|
Feb 21, 2009
|
Nov 08, 2009
|
|
|
CONF-14948
|
Support failover NICs for cluster configuration...
|
Unassigned
|
Tony Atkins [Atlassian]
|
|
Open
|
Unresolved
|
Mar 19, 2009
|
Mar 19, 2009
|
|
|
CONF-15523
|
Run cluster performance build on two machines
|
Unassigned
|
Matt Ryall [Atlassian]
|
|
Open
|
Unresolved
|
May 05, 2009
|
May 12, 2010
|
|
|
CONF-16419
|
Installing a font for PDF export in a cluster will not carry to cluster nodes that are down or unavailable.
|
Unassigned
|
Charles Miller [Atlassian]
|
|
Open
|
Unresolved
|
Jul 20, 2009
|
Aug 05, 2009
|
|
|
CONF-17089
|
Reindexing in cluster only runs on one node if triggered from web UI
|
Unassigned
|
Anatoli Kazatchkov [Atlassian]
|
|
Open
|
Unresolved
|
Oct 01, 2009
|
May 12, 2010
|
|
|
CONF-18241
|
Uninstalling a plugin from a cluster sometimes fails
|
Unassigned
|
Don Willis [Atlassian]
|
|
Open
|
Unresolved
|
Jan 13, 2010
|
Apr 13, 2010
|
|
|
CONF-19158
|
Frequent logins by one user across a cluster lead to errors
|
Unassigned
|
Don Willis [Atlassian]
|
|
Open
|
Unresolved
|
Mar 29, 2010
|
Mar 30, 2010
|
|
|
CONF-9335
|
In cluster, allow attachments to be stored on file system in network-shared directory
|
Unassigned
|
Jeremy Largman [Atlassian]
|
|
Open
|
Unresolved
|
Aug 29, 2007
|
May 11, 2010
|
|
|
CONF-17040
|
Cannot build milestones outside Atlassian due to coherence
|
Unassigned
|
Jonathan Gilbert [Atlassian]
|
|
Open
|
Unresolved
|
Sep 25, 2009
|
Dec 08, 2009
|
|
|
CONF-7368
|
Confluence Cluster setup dies horribly when DNS fails
|
Unassigned
|
Don Willis [Atlassian]
|
|
Open
|
Unresolved
|
Nov 23, 2006
|
Jan 19, 2009
|
|
|
CONF-9020
|
Cluster nodes do not get notified of Layout changes
|
Unassigned
|
Roberto Dominguez
|
|
Open
|
Unresolved
|
Jul 27, 2007
|
Oct 08, 2007
|
|
|
CONF-9059
|
Cluster setup network interface selection shows loopback interface
|
Unassigned
|
Matt Ryall [Atlassian]
|
|
Open
|
Unresolved
|
Aug 01, 2007
|
Aug 01, 2007
|
|
|
CONF-9281
|
Plugin's I18n properties not loaded in other cluster nodes unless restarted
|
Unassigned
|
Roberto Dominguez
|
|
Open
|
Unresolved
|
Aug 26, 2007
|
Feb 15, 2010
|
|
|
CONF-9749
|
Coherence does not allow the disabling of all JDK shutdown hooks
|
Unassigned
|
Christopher Owen [Atlassian]
|
|
Open
|
Unresolved
|
Oct 17, 2007
|
Jan 29, 2008
|
|
|
CONF-10635
|
Database logging of clustersafety access
|
Unassigned
|
James Fleming [Atlassian]
|
|
Open
|
Unresolved
|
Feb 03, 2008
|
Feb 03, 2008
|
|
|
CONF-10953
|
Support unicast addressing in cluster when well-known-addresses WKA are defined
|
Unassigned
|
Ivan Benko [Atlassian]
|
|
Open
|
Unresolved
|
Mar 06, 2008
|
Aug 25, 2009
|
|
|
CONF-10977
|
Generate new Multicast address from a "new" cluster name
|
Unassigned
|
Ivan Benko [Atlassian]
|
|
Open
|
Unresolved
|
Mar 06, 2008
|
Sep 11, 2008
|
|
|
CONF-10979
|
List confluence cluster interface
|
Unassigned
|
Ivan Benko [Atlassian]
|
|
Open
|
Unresolved
|
Mar 06, 2008
|
Mar 06, 2008
|
|
|
CONF-10981
|
Check how many nodes/processes running in a cluster and their identity
|
Unassigned
|
Ivan Benko [Atlassian]
|
|
Open
|
Unresolved
|
Mar 06, 2008
|
Mar 06, 2008
|
|
|
CONF-11206
|
Confluence Clustered and JIRA trust delegation
|
Unassigned
|
Ivan Benko [Atlassian]
|
|
Open
|
Unresolved
|
Mar 25, 2008
|
May 12, 2010
|
|
|
CONF-12421
|
Don't use distrbuted cache for storing Captcha's in a cluster
|
Unassigned
|
Paul Curren [Atlassian]
|
|
Open
|
Unresolved
|
Jul 13, 2008
|
Jul 13, 2008
|
|
|
CONF-13698
|
Changing custom html on one node of a cluster is not imideatly reflected on the other node.
|
Unassigned
|
Anatoli Kazatchkov [Atlassian]
|
|
Open
|
Unresolved
|
Nov 12, 2008
|
Nov 13, 2008
|
|
|
CONF-13870
|
After a site Import into a cluster, admin console displays attachment storage as filesystem
|
Unassigned
|
Agnes Ro [Atlassian]
|
|
Open
|
Unresolved
|
Nov 27, 2008
|
Dec 03, 2008
|
|
|
CONF-14338
|
Specify an arbitrary multicast port for a cluster
|
Unassigned
|
James Fleming [Atlassian]
|
|
Open
|
Unresolved
|
Jan 30, 2009
|
Jan 30, 2009
|
|
|
CONF-17577
|
Cluster build passed but didn't close down Confluence
|
Unassigned
|
Brian Nguyen [Atlassian]
|
|
Open
|
Unresolved
|
Nov 10, 2009
|
May 12, 2010
|
|
|
CONF-19559
|
Provide support for Confluence clustered in a virtualized environment...
|
Unassigned
|
Tony Atkins [Atlassian]
|
|
Open
|
Unresolved
|
May 06, 2010
|
May 16, 2010
|
|
|
CONF-19626
|
Support more than four clustered nodes
|
Unassigned
|
Tony Atkins [Atlassian]
|
|
Open
|
Unresolved
|
May 12, 2010
|
May 16, 2010
|
|
|
CONF-9712
|
Plugins which don't work in a cluster shouldn't look like an error
|
Unassigned
|
Gary Weaver
|
|
Open
|
Unresolved
|
Oct 15, 2007
|
Oct 16, 2007
|
|
|
CONF-9846
|
Tangosol configuration: the (optional) cluster-name element is in the wrong place
|
Unassigned
|
Don Willis [Atlassian]
|
|
Open
|
Unresolved
|
Oct 30, 2007
|
Dec 20, 2007
|
|
|
CONF-14088
|
Locking on cache keys needs to check if the lock was actually aquired
|
Unassigned
|
Chris Kiehl [Atlassian]
|
|
Open
|
Unresolved
|
Dec 30, 2008
|
Nov 22, 2009
|
|
|
CONF-8300
|
Cannot override TTL configuration through tangosol coherence properties
|
Unassigned
|
Matthew Jensen [Atlassian]
|
|
Needs Verification
|
Unresolved
|
Apr 20, 2007
|
Nov 10, 2009
|
|
|
CONF-10330
|
Certain Multicast IP addresses sending CHANGE_TO_EXCLUDE_MODE causing IGMP traffic to be blocked
|
Unassigned
|
Partha Kamal [Atlassian]
|
|
Needs Verification
|
Unresolved
|
Dec 28, 2007
|
Nov 26, 2008
|
|
Contact Atlassian support
We have dedicated staff on hand to support your installation of Confluence. Please follow the instructions for raising a support request and mention that you're having trouble setting up your Confluence cluster.
Related
Cluster Safety Mechanism
|