summaryrefslogtreecommitdiff
path: root/doc/vmxnet3-usermap.html
blob: d15a0f4d8b4841bcbdf4f68ca90c8ed5c73d9e6b (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
<!DOCTYPE html>
<html lang="en">
	<head>
		<meta charset="utf-8" />
		<meta name=viewport content="width=device-width, initial-scale=1" />
		<title>DPDK doc</title>
		<link rel="stylesheet" href="/reset.css" />
		<link rel="stylesheet" href="/layout.css" />
		<link rel="stylesheet" href="/content.css" />
		<script type="text/javascript">
			var _gaq = _gaq || [];
			_gaq.push(['_setAccount', 'UA-39229303-1']);
			_gaq.push(['_setDomainName', 'dpdk.org']);
			_gaq.push(['_setAllowLinker', true]);
			_gaq.push(['_trackPageview']);
			(function() {
				var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
				ga.src = '//www.google-analytics.com/ga.js';
				var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
			})();
		</script>
	</head>
	<body>
		<header>
			<h1><a href="//dpdk.org/"><img src="/logos/dpdk.png"
				alt="DPDK: Data Plane Development Kit"></a></h1>
			<nav>
				<ul>
					<li><a href="/">Home</a></li>
					<li><a href="/download">Download</a></li>
					<li><a href="/doc" id="current">Documentation</a></li>
					<li><a href="/dev">Development</a></li>
					<li><a href="/ml">Mailing Lists</a></li>
					<li><a href="/events">Events</a></li>
					<li><a href="/news">News</a></li>
					<li><a href="/about">About</a></li>
				</ul>
			</nav>
		</header>
		<section>
<h2>vmxnet3-usermap</h2>

<!-- Copyright 2013 6WIND S.A.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
  notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright
  notice, this list of conditions and the following disclaimer in
  the documentation and/or other materials provided with the
  distribution.
- Neither the name of 6WIND S.A. nor the names of its
  contributors may be used to endorse or promote products derived
  from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
OF THE POSSIBILITY OF SUCH DAMAGE. -->
<h3>DESCRIPTION</h3>
<p>Here are two drivers for the third generation of paravirtualized NIC in
VMware ESXi (vmxnet3): a Linux kernel driver and a Poll-Mode Driver (PMD)
for DPDK. They work together.</p>
<p>The standard Linux kernel driver for this vNIC is simply called vmxnet3.ko.
The &quot;kmod&quot; subdirectory provides a replacement called vmxnet3-usermap.ko.
It essentially provides a new mode called usermap using the enable_shm=2
option.</p>
<p>The DPDK driver found in the &quot;pmd&quot; subdirectory is a shared object
(librte_pmd_vmxnet3.so) that must be loaded as a DPDK extension.
This driver gets hardware access through the usermap mode of the kernel
driver.</p>
<h3>HOW TO BUILD</h3>
<p>There are two build dependencies: Linux kernel headers and DPDK &gt;= 1.2.3r3.</p>
<p>The kernel driver is based on vmxnet3-8.6.10 from VMware ESXi-5.0.0u2.
So the latest supported Linux version is 3.2.</p>
<p>DPDK must be built first in order to have all its header files in the same
directory, that RTE_INCLUDE must point to.</p>
<p>Example:</p>
<pre>
dpdk$ make config T=x86_64-native-linuxapp-gcc
[...]
dpdk$ make
[...]
dpdk$ cd ../vmx3
vmx3$ make RTE_INCLUDE=../dpdk/build/include
[...]
</pre>
<p>Build options can be displayed using <span class="pre">make help</span>.</p>
<h3>HOW TO RUN</h3>
<p>Previous driver must be unloaded:</p>
<pre>
$ rmmod vmxnet3
</pre>
<p>The driver must be loaded in usermap mode:</p>
<pre>
$ insmod vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=512 num_txds=512
</pre>
<p>The above command will enable usermap-mode only for the first two vmxnet3
interfaces detected. Commas can be used to separate parameters values if
several interfaces need to be managed from userland.</p>
<p>The following parameters take one value per interface:</p>
<ul class="simple">
<li><span class="pre">enable_shm=2[,2[...]</span>: configure usermap mode suitable for DPDK use
(default is 0 for kernel mode)</li>
<li><span class="pre">num_rqs=int[,int[...]</span>: number of RX queues
(default is the number of detected CPUs)</li>
<li><span class="pre">num_tqs=int[,int[...]</span>: number of TX queues
(default is the number of detected CPUs)</li>
</ul>
<p>Other parameters have a global effect:</p>
<ul class="simple">
<li><span class="pre">num_rxds=int</span>: number of RX descriptors per queue</li>
<li><span class="pre">num_txds=int</span>: number of TX descriptors per queue</li>
</ul>
<p>Because the kernel module initializes interfaces, <strong>it's very important that
DPDK parameters match kernel ones</strong>. It won't work otherwise.</p>
<p>For more information, use <span class="pre">modinfo <span class="pre">vmxnet3-usermap.ko</span>.</p>
<p>In userland, librte_pmd_vmxnet3.so must be loaded by the DPDK using the <span class="pre">-d</span>
option.</p>
<p>Another interesting option is <span class="pre">--vmware-tsc-map</span>.
It allows VMware TSC mapping which provides an access to physical TSC.
When this option is used, take care of setting this ESXi parameter:</p>
<pre>
monitor_control.pseudo_perfctr = TRUE
</pre>
<p><span class="pre">testpmd</span> example for two ports, matching the <span class="pre">insmod</span> command above:</p>
<pre>
$ ../dpdk/build/app/testpmd -c 0xff -n 1 \
  -d librte_pmd_vmxnet3.so \
  -- \
  -i --rxq=1 --rxd=512 --txd=512
[...]
EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
Interactive-mode selected
Initializing port 0... done:  Link Up - speed 10000 Mbps - full-duplex
Initializing port 1... done:  Link Up - speed 10000 Mbps - full-duplex
testpmd&gt;
</pre>
<h3>HOW TO RESTART APPLICATIONS</h3>
<p>In order to relaunch a DPDK application using vmxnet3-usermap,
the kernel driver must be reinitialized:</p>
<pre>
$ rmmod vmxnet3_usermap
$ insmod vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=512 num_txds=512
</pre>
<p>Then the DPDK application can be restarted.</p>
		</section>
		<footer></footer>
	</body>
</html>