summaryrefslogtreecommitdiff
path: root/README.md
blob: e1085e08e7a1d2c65acdcc168e2234853ffa4f53 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
# Soft Patch Panel


## Overview

[Soft Patch Panel](http://git.dpdk.org/apps/spp/)
(SPP) is a DPDK application for providing switching
functionality for Service Function Chaining in NFV
(Network Function Virtualization).
DPDK stands for Data Plane Development Kit.


## Project Goal

In general, implementation and configuration of DPDK application is
difficult because it requires deep understandings of system architecture
and networking technologies.

The goal of SPP is to easily inter-connect DPDK applications together
and assign resources dynamically to these applications
with patch panel like simple interface.


## Architecture Overview

SPP is a kind of DPDK
[multi-process application](https://doc.dpdk.org/guides/prog_guide/multi_proc_support.html).
It is composed of primary process which is responsible for resource
management, and secondary processes as workers for packet forwarding.
This primary process does not interact with any traffic. It is
responsible for creation and freeing of resources shared among other
secondary processes.

A Python based management interface called SPP controller consists of
`spp-ctl` and `SPP CLI`.
`SPP CLI` is a command line interface of SPP, and `spp-ctl` a backend
server for managing primary and secondary processes.
`spp-ctl` behaves as a REST API server and `SPP CLI` sends a command via
the REST API.


## Install

Before using SPP, you need to install DPDK. Briefly describ here how to
install and setup DPDK. Please refer to SPP's
[Getting Started](https://doc.dpdk.org/spp/setup/getting_started.html) guide
for more details. For DPDK, refer to
[Getting Started Guide for Linux](https://doc.dpdk.org/guides/linux_gsg/index.html).

It is required to install Python3 and libnuma-devel library before.
SPP does not support Python2 anymore.

```sh
$ sudo apt install python3 python3-pip
$ sudo apt install libnuma-dev
```

### DPDK

Clone repository and compile DPDK in any directory.

```
$ cd /path/to/any
$ git clone http://dpdk.org/git/dpdk
```

Compile DPDK with target environment.

```sh
$ cd dpdk
$ export RTE_SDK=$(pwd)
$ export RTE_TARGET=x86_64-native-linuxapp-gcc  # depends on your env
$ make install T=$RTE_TARGET
```

### SPP

Clone repository and compile SPP in any directory.

```sh
$ cd /path/to/any
$ git clone http://dpdk.org/git/apps/spp
$ cd spp
$ make  # Confirm that $RTE_SDK and $RTE_TARGET are set
```

### Binding Network Ports to DPDK

Network ports must be bound to DPDK with a UIO (Userspace IO) driver.
UIO driver is for mapping device memory to userspace and registering
interrupts.

You usually use the standard `uio_pci_generic` for many use cases or
`vfio-pci` for more robust and secure cases. Both of drivers are
included by default in modern Linux kernel.

```sh
# Activate uio_pci_generic
$ sudo modprobe uio_pci_generic

# or vfio-pci
$ sudo modprobe vfio-pci
```

Once UIO driver is activated, bind network ports with the driver.
DPDK provides `usertools/dpdk-devbind.py` for managing devices.
Here are some examples.

```
# Bind a port with 2a:00.0 (PCI address)
$ ./usertools/dpdk-devbind.py --bind=uio_pci_generic 2a:00.0

# or eth0
$ ./usertools/dpdk-devbind.py --bind=uio_pci_generic eth0
```

After binding a port, you can find it is under the DPDK driver,
and cannot find it by using `ifconfig` or `ip`.

```sh
$ $RTE_SDK/usertools/dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
0000:2a:00.0 '82571EB ... 10bc' drv=uio_pci_generic unused=vfio-pci
....
```

## How to Use

You can use SPP from `bin/start.sh` script or launching each of
processes manually. This startup script is provided for skipping to
input many options for multiple DPDK applications.

### Quick Start

Start with `bin/start.sh` with configuration file `bin/config.sh`.
First time you run the startup script, it generates the config file
and asks to edit the config without launchin processes.

Here is a part of config parameters. You do not need to change most of
params.
If you do not have physical NICs on your server, activate
`PRI_VHOST_IDS` which is for setting up vhost interfaces instead of
physical.

```sh
SPP_HOST_IP=127.0.0.1
SPP_HUGEPAGES=/dev/hugepages

# spp_primary options
LOGLEVEL=7  # change to 8 if you refer debug messages.
PRI_CORE_LIST=0  # required one lcore usually.
PRI_MEM=1024
PRI_MEMCHAN=4  # change for your memory channels.
NUM_RINGS=8
PRI_PORTMASK=0x03  # total num of ports of spp_primary.
#PRI_VHOST_IDS=(11 12)  # you use if you have no phy ports.
```

After you edit configuration, you can launch `spp-ctl`,
`spp_primary` and `SPP CLI` from startup script.

```sh
$ ./bin/start.sh
Start spp-ctl
Start spp_primary
Waiting for spp-ctl is ready ...
Welcome to the SPP CLI. Type `help` or `?` to list commands.

spp >
```

Check status of `spp_primary` because it takes several seconds to be
ready. Confirm that the status is `running`.

```sh
spp > status
- spp-ctl:
  - address: 127.0.0.1:7777
- primary:
  - status: running
- secondary:
  - processes:
```

Now you are ready to launch secondary processes from `pri; launch`
command, or another terminal.
Here is an example for launching `spp_nfv` with options from
`pri; launch`. Log file of this process is created as
`log/spp_nfv1.log`.

```sh
spp > pri; launch nfv 1 -l 1,2 -m 512 -- -n 1 -s 127.0.0.1:6666
```

This `launch` command supports TAB completion. Parameters for `spp_nfv`
are completed after secondary ID `1`.

```sh
spp > pri; launch nfv 1

# Press TAB
spp > pri; launch nfv 1 -l 1,2 -m 512 -- -n 1 -s 127.0.0.1:6666
```

It is same as following options launching from terminal.

```sh
$ sudo ./src/nfv/x86_64-native-linuxapp-gcc/spp_nfv \
    -l 1,2 -n 4 -m 512 \
    --proc-type secondary \
    -- \
    -n 1 \
    -s 127.0.0.1:6666
```

Parameters for completion are defined in `SPP CLI`, and you can find
parameters with `config` command.

```sh
spp > config
- max_secondary: "16"   # The maximum number of secondary processes
- prompt: "spp > "  # Command prompt
- topo_size: "60%"  # Percentage or ratio of topo
- sec_mem: "-m 512" # Mem size
...
```

You can launch consequence secondary processes from CLI.

```sh
spp > pri; launch nfv 2 -l 1,3 -m 512 -- -n 2 -s 127.0.0.1:6666
spp > pri; launch vf 3 -l 1,4,5,6 -m 512 -- -n 3 -s 127.0.0.1:6666
...
```

### Startup Manually

You should keep in mind the order of launching processes if you launch
processes without using startup script.
`spp-ctl` should be launched before all of other processes. Then,
primary process must be launched before secondary processes.
On the other hand, you can launch SPP CLI `spp.py` in any time
after `spp-ctl`. In general, `spp-ctl` is launched first,
then `spp.py`, `spp_primary` and secondary processes.

It has a option -b for binding address explicitly to be accessed from
other than default, `127.0.0.1` or `localhost`.

In the following example, processes are launched in different terminals
for describing options, although you can launch them with
`pri; launch` command.

#### SPP Controller

SPP controller consists of `spp-ctl` and SPP CLI.
`spp-ctl` is a HTTP server for REST APIs for managing SPP processes.

```sh
# terminal 1
$ python3 src/spp-ctl/spp-ctl -b 192.168.1.100
```

SPP CLI is a client of `spp-ctl` for providing simple user interface
without using REST APIs.

```sh
# terminal 2
$ python3 src/spp.py -b 192.168.1.100
```

#### SPP Processes

Launch SPP primary and secondary processes.
SPP primary is a resource manager and initializing EAL for secondary
processes. Secondary process behaves as a client of primary process
and a worker for doing tasks.

```sh
# terminal 3
$ sudo ./src/primary/x86_64-native-linuxapp-gcc/spp_primary \
    -l 1 -n 4 \
    --socket-mem 512,512 \
    --huge-dir=/dev/hugepages \
    --proc-type=primary \
    -- \
    -p 0x03 \
    -n 10 \
    -s 192.168.1.100:5555
```

There are several kinds of secondary process. Here is an example of the simplest
one.

```sh
# terminal 4
$ sudo ./src/nfv/x86_64-native-linuxapp-gcc/spp_nfv \
    -l 2-3 -n 4 \
    --proc-type=secondary \
    -- \
    -n 1 \
    -s 192.168.1.100:6666
```

After all of SPP processes are launched, configure network path from SPP CLI.
Please refer to SPP
[Use Cases](https://doc.dpdk.org/spp/setup/use_cases.html)
for the configuration.