I recently faced a network-related problem, where TCP communications between two servers were particularly slow.
A Customer of mine is running both SugarCRM and MySql on the same (Windows Server 2008) machine. Since they’re collecting a large number of records and a huge number of attachments, I suggested them to move MySql to a new Linux Server, to ease the Windows Server. So we took an amazing HP Server with two esacore processors and 12GB of RAM, running Ubuntu Server 10.04 LTS x64 : not bad, isn’t it ?
After the initial setup we changed the test environment, making it point to the new database server and…surprise !! It’s significantly slower than the production !
We solved the problem changing a bunch of recurring procedures we built on top of standard Sugar’s objects, substituting them with direct queries on the database.
Clear enough, the main difference is the Network latency: every query issued by SugarCRM now have a small amount of time around itself… and if Your pages are heavily dealing with the database, You better avoid standard SugarCRM objects.
Ok, but can a 1Gb connection really kill performances in this way ?
To answer this question i made some test with iperf, a Network Performance Testing Tool available for both Linux (it’s in the standard ubuntu and CentOS repositories – just apt-get install iperf) and Windows (You can download xjperf from here – remember You’ll need JRK installed, since it’a GUI Java program). Test a Network Connection between two machines with iperf is really simple.
On the first machine, that we’ll call Server, You need to issue the following command:
iperf -s
On the second machine, the Client, You have to issue the following command:
iperf -c <ipaddress>
(where <ipaddress> is the Server machine’s IP Address – or a computer name, if resolved by a DNS).
In this way You’ll test the speed of TCP data generated by the Client for the Server. Inverting the commands, You can test the flow in the opposite direction. If using xjperf, You can do the same through the GUI. Of course You can do the same test between iperf and xjperf: this would be the case of a test between a Linux machine and a Windows machine.
I made a number of tests between a bunch of machines running Linux (Ubuntu 10.04, CentOS) and Windows (XP, Seven, Server 2008), on both 100M and 1G Network Connections. Tests were made both with a Cross UTP5e cable directly connecting the machines and a connection through a dedicated switch (actually there’s no difference). The following table shows the range of speeds I reached (in % of bandwidth):
- Linux to Linux : 90% – 95%
- Linux to Windows : 80% – 89%
- Windows to Linux : 40% – 60%
- Windows to Windows : 30% – 40%
The first test I made was Linux to Linux. The second one was Linux to Windows. The third one was Windows to Linux. At this point I was stuck : I repeated it many times, always with the same results. I couldn’t believe at them !
Finally I’ve found the trick: it’s named “TCP Window Size”. Watching the iperf log, You can see its value: in my case was 85Kb for Linux and 8/16Kb for Windows. Increasing it to 56Kb (iperf let You do this temporary change) on Windows side, bandwidth occupation goes straight to 80%-90% !
Unfortunately this is not a definitive solution: starting from Windows Vista this parameter (formerly in the registry) is no more customizable.
What about Your machines ? Will You try this simple test ? If so, will You share Your results/considerations ?
I finally did it. After the introductory course (courtesy of Cristiano, member of GlugTo) I seriously started playing with Nagios.
