Abstract
Netperf is a network performance benchmarking tool that can be used to measure the data throughput rate of both TCP and UDP communications across the network. This article will describe the steps necessary to
properly install and setup Netperf on Linux.
Introduction
Network performance monitoring is critical for today's high performance computing clusters. Many HPC
Linux clusters require the data transfer rate of installed network adapters to transmit data at the speed at which
they were designed to perform (i.e., 100Mb/s, 1000Mb/s, etc). Unfortunately, in the absence of a network
benchmarking tool, the node-to-node network performance of these adapters cannot be properly determined.
The downfall to this is that manufacturer specific tunable parameters for network adapters and operating
system's performance metrics will unequivocally go unused resulting in continued poor network performance.
In addition, the operating system's network protocols metrics might be configured to use minimum/default
values causing the system to bottleneck.
Obtaining and Installing Netperf
Netperf can be obtained from http://www.netperf.org/netperf/. The latest version, currently at 2.4.3, is recommended for networks configured to use IPv4 and IPv6. Once you have obtained the Netperf source code, extract it in a temporary directory then build the binaries. The following steps can be used to accomplish this task:
1. Download Netperf to a staging area on your Linux system
2. Unzip and untar the Netperf compressed source file
#tar -xzvf netperf-x.x.x.tar.gz, where x.x.x is the current version number.
3. Change to the directory where the Netperf source files were extracted (i.e., cd netperf-x.x.x, where x.x.x
is the version number)
4. Run ./configure
5. Run make
6. Run 'make install' to install the Netperf program binaries in /usr/local/bin. Note you must be logged in as root to write to the /usr/local/bin directory
Preparing to use Netperf
Netperf can be run as a standalone daemon or installed as a service daemon in the /etc/xinetd.d
directory. Two files are created when the Netperf source is compiled. The first is the Netperf server
'netserver', which must be run as a daemon in order to measure data throughput. The second 'netperf'
which is the client-side program that is used to communicate with the Netperf server program. The Netperf
client program sends the streams of data to the Netperf server unidirectionally and reports the rate of transfer
back to the user.
To run the Netperf as a standalone daemon, simply invoke the 'netserver' program then run 'netperf' to
observe the rate of transfer. For example, to see the data throughput on the node running both the server and
client program, do the following:
1. Invoke the Netperf server program
$ netserver
Starting netserver at port 12865
Starting netserver at hostname 0.0.0.0 port 12865 and family
AF_UNSPEC
2. Run the Netperf client program and observe the output
$ netperf
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
localhost (127.0.0.1) port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
87380 87380 87380 10.01 634.75
Notice that the maximum data throughput is approximately 63 MB/s when the client program is run on the
same node as the server. For a more detailed description on interpreting the output, refer to the Netperf manual at http://www.netperf.org/svn/netperf2/tags/netperf-2.4.3/doc/netperf.html.
In order to run Netperf as a server daemon you must be logged in as root to edit the /etc/services file
and to add a service file in the /etc/xinetd.d/ directory. Assuming your are logged in as root, edit the
/etc/services file and add the entry such as the following to the end of the file.
# Add to end of /etc/services
netperf 12865/tcp # Network performance monitoring
netperf 12865/udp # Network performance monitoring
Next change to the /etc/xinetd.d directory and create a service daemon file call netperf with the
following content:
# Netperf server program service daemon
service netperf
{
disable = no
socket_type = stream
protocol = tcp
wait = no
user = root
server = /usr/local/bin/netserver
}
Restart the xinetd daemon for the Netperf server program to be added as a service daemon.
To test for communication between the Netperf service daemon and a remote client do the following:
1. Ensure that the Netperf client program code is available on the node from which it is to be invoked
2. Run the Netperf client program on the node with the -H option
$ netperf -H <server>, where server is the name of the machine running the Netperf server
service daemon. For example:
$ netperf -H tracker
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
tracker.mydomain.com (172.168.0.1) port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
87380 87380 87380 10.02 92.04
Again, notice that maximum data transfer rate is approximately 9.2 MB/s. Also, the default receive socket size as set on the Netperf server is 87380 bytes while the send socket and send message size are equivalent. For a more detailed description of the flags that can be used with Netperf plus other examples, please refer to the Netperf manual located at http://www.netperf.org/svn/netperf2/tags/netperf-2.4.3/doc/netperf.html.
Summary
This article has demonstrated how Netperf can be used as a network benchmark tool for evaluating the
performance of your HPC network. Not only will you find the tool useful, but it can help you to decide what
settings should be passed to the Linux operating system TCP send and receive socket buffers to help increase data throughput for node-to-node communications.
0 comments
Post a Comment