summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJonathan Ribas <jonathan.ribas@fraudbuster.mobi>2019-06-28 17:27:04 +0200
committerJonathan Ribas <jonathan.ribas@fraudbuster.mobi>2019-06-28 17:27:04 +0200
commit902807072ac88b4f1e631a22373555e0c0561141 (patch)
tree1b86f5f1ab12e823361f4c8f3c1be6c50b887afe
parentc14c6390c785ade94a409564ba6ce22153c44bd9 (diff)
parentfb047306061c2b219abb523ddca8a1b015077c74 (diff)
downloaddpdk-burst-replay-902807072ac88b4f1e631a22373555e0c0561141.zip
dpdk-burst-replay-902807072ac88b4f1e631a22373555e0c0561141.tar.gz
dpdk-burst-replay-902807072ac88b4f1e631a22373555e0c0561141.tar.xz
Merge branch 'wip-doc'
-rw-r--r--docs/src/contact.rst22
-rw-r--r--docs/src/how-to-contribute.rst39
-rw-r--r--docs/src/index.rst9
-rw-r--r--docs/src/introduction.rst106
-rw-r--r--docs/src/quick-start-guide.rst120
-rw-r--r--docs/src/readme.rst68
-rw-r--r--docs/src/todo-list.rst18
-rw-r--r--docs/src/user-guide.rst104
8 files changed, 416 insertions, 70 deletions
diff --git a/docs/src/contact.rst b/docs/src/contact.rst
new file mode 100644
index 0000000..c2af920
--- /dev/null
+++ b/docs/src/contact.rst
@@ -0,0 +1,22 @@
+.. dpdk-burst-replay: BSD-3-Clause
+ Copyright 2018 Jonathan Ribas, FraudBuster. All rights reserved.
+
+.. _contact:
+
+Contact
+=======
+
+Emails
+------
+
+Please use the users@dpdk.org and dev@dpdk.org mailing lists and specify
+burst-replay on the subject.
+
+Here are my personal email addresses:
+
+jonathan.ribas@fraudbuster.mobi / contact.jonathanribas@gmail.com
+
+IRC
+---
+
+#dpdk-burst-replay on freenode is the official dedicated place to chat.
diff --git a/docs/src/how-to-contribute.rst b/docs/src/how-to-contribute.rst
new file mode 100644
index 0000000..54cace8
--- /dev/null
+++ b/docs/src/how-to-contribute.rst
@@ -0,0 +1,39 @@
+.. dpdk-burst-replay: BSD-3-Clause
+ Copyright 2018 Jonathan Ribas, FraudBuster. All rights reserved.
+
+.. _how-to-contribute:
+
+How to contribute
+=================
+
+**THE MOST IMPORTANT**: do not hesitate to do it. :)
+
+Bugzilla
+--------
+
+If you:
+
+* Find a bug in the application.
+* Want a feature to be developed.
+* Think that the documentation isn't clear enough.
+* Have any suggestion.
+
+Go first fill a new bug in `Bugzilla <https://bugs.dpdk.org/describecomponents.cgi?product=dpdk-burst-replay>`_. It will let me (and others) discuss it and synchronise our work.
+
+Get the code
+------------
+
+URL to git clone::
+
+ git://dpdk.org/apps/dpdk-burst-replay
+ http://dpdk.org/git/apps/dpdk-burst-replay
+
+To browse the code online: https://git.dpdk.org/apps/dpdk-burst-replay/
+
+Sending patchs
+--------------
+
+To send patches, please follow the `official DPDK guide to contribution <https://www.dpdk.org/contribute/>`_, whith these little differences:
+
+* Add **[burst-replay PATCH]** to the title (instead of just **[PATCH]**).
+* Add me in CC: jonathan.ribas@fraudbuster.mobi.
diff --git a/docs/src/index.rst b/docs/src/index.rst
index 16ad52e..e39408b 100644
--- a/docs/src/index.rst
+++ b/docs/src/index.rst
@@ -7,6 +7,11 @@ dpdk-burst-replay's documentation
Contents:
.. toctree::
- :maxdepth: 1
+ :maxdepth: 2
- readme
+ introduction
+ quick-start-guide
+ user-guide
+ how-to-contribute
+ todo-list
+ contact
diff --git a/docs/src/introduction.rst b/docs/src/introduction.rst
new file mode 100644
index 0000000..570e942
--- /dev/null
+++ b/docs/src/introduction.rst
@@ -0,0 +1,106 @@
+.. dpdk-burst-replay: BSD-3-Clause
+ Copyright 2019 Jonathan Ribas, FraudBuster. All rights reserved.
+
+.. _introduction:
+
+Introduction
+============
+
+The tool is designed to provide high DPDK performances to burst any pcap dump on
+a single or multiple NIC port(s), while keeping it simple to use.
+
+It's a Linux command line tool, wrote in C and compiling on both x86_64 and arm64.
+It mainly use DPDK, but rely also on libnuma, which are the only tool dependencies.
+
+The tool usage tend to looks like tcpreplay in term of simplicity. Only two args are required:
+one pcap file and one DPDK port address to replay on:
+
+::
+
+ $> dpdk-replay foo.pcap 04:00.0
+
+What can it do?
+---------------
+
+* Load a pcap file: standard input format for all common dumping/analyzing/replaying tools like
+ tcpdump, tcpreplay, wireshark etc...
+* Put all loaded packets in cache(s) to avoid reading the pcap file while its been replays.
+* Burst packets simultaneously on multiple ports (ATM, only NICs on the same NUMA can be addressed).
+* Run the same pcap several times in a row. It can be useful to extend a stress test without having
+ to provide a bigger pcap file.
+* At the end, some stats are displayed for each ports (time spent, pkts-per-sec, total bitrate etc...)
+* Abstract DPDK stack: All EAL/mempool/ports initializations are automatically handled.
+
+.. _dpdk_compatible_versions:
+
+DPDK compatible versions
+------------------------
+
+dpdk-burst-replay releases are compatible with DPDK LTS versions.
+
++----------------------------+---------------------------+
+| dpdk-burst-replay releases | Tested with DPDK versions |
++============================+===========================+
+| v1.1.0 | 16.11.9/17.11.5/18.11.1 |
++----------------------------+---------------------------+
+
+NB: Other intermediate versions should also works. Last versions can need some adaptation.
+
+Ok, let's start!
+----------------
+
+If you want try it, please have a look on the :ref:`quick-start-guide` and :ref:`user-guide`:
+
+.. toctree::
+ :maxdepth: 2
+
+ quick-start-guide
+ user-guide
+
+If you're looking for the GIT, have a look `here <https://git.dpdk.org/apps/dpdk-burst-replay/>`_.
+
+And if you find a bug, want to share a suggestion or see what are the opened issues,
+please check the `Bugzilla <https://bugs.dpdk.org/describecomponents.cgi?product=dpdk-burst-replay>`_.
+
+
+FOSDEM 2019 presentation
+------------------------
+
+A presentation have been made at FOSDEM'19. You can find bellow the event info, slides and video links:
+
+* Event info: `FOSDEM site <https://fosdem.org/2019/schedule/event/dpdk_burst_replay/>`_.
+* Video: `FOSDEM <https://video.fosdem.org/2019/H.2214/dpdk_burst_replay.mp4>`_ or `Youtube <https://www.youtube.com/watch?v=j56Y-SPRDE0>`_.
+* Slides: `PDF file <https://fosdem.org/2019/schedule/event/dpdk_burst_replay/attachments/slides/2874/export/events/attachments/dpdk_burst_replay/slides/2874/presentation.pdf>`_.
+
+
+BSD Licence
+-----------
+
+BSD 3-Clause License
+
+Copyright (c) 2018, Jonathan Ribas, FraudBuster. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+ list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+ this list of conditions and the following disclaimer in the documentation
+ and/or other materials provided with the distribution.
+
+* Neither the name of the copyright holder nor the names of its
+ contributors may be used to endorse or promote products derived from
+ this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
diff --git a/docs/src/quick-start-guide.rst b/docs/src/quick-start-guide.rst
new file mode 100644
index 0000000..41bb269
--- /dev/null
+++ b/docs/src/quick-start-guide.rst
@@ -0,0 +1,120 @@
+.. dpdk-burst-replay: BSD-3-Clause
+ Copyright 2018 Jonathan Ribas, FraudBuster. All rights reserved.
+
+.. _quick-start-guide:
+
+Quick start guide
+=================
+
+Install dependencies
+--------------------
+
+With your distro package manager, install these packages (names can change a bit):
+
+* dpdk-dev (obsly)
+* libnuma-dev
+
+For Debian::
+
+ sudo apt-get install dpdk-dev libnuma-dev
+
+NB: libpcap is not required, as dpdk-replay process pcap files manually.
+
+To manually compile with a custom DPDK version, see :ref:`installing_custom_dpdk_version`.
+
+Compiling and installing it
+---------------------------
+
+Standard way, using autotools::
+
+ autoreconf -i && ./configure && make && sudo make install
+
+To use the static Makefile with your custom DPDK version, see :ref:`linking_custom_dpdk_version`.
+
+Binding wanted DPDK ports
+-------------------------
+
+First, you need to unbind the ports you want to use from the Kernel to bind them to DPDK.
+
+To list the available ports::
+
+ $> dpdk-devbind.py --status
+
+ [...]
+ Network devices using DPDK-compatible driver
+ ============================================
+ <none>
+
+ Network devices using kernel driver
+ ===================================
+ 0000:01:00.0 'NetXtreme BCM5720 Gigabit Ethernet PCIe 165f' if=eth0 drv=tg3 unused=igb_uio *Active*
+ 0000:01:00.1 'NetXtreme BCM5720 Gigabit Ethernet PCIe 165f' if=eth1 drv=tg3 unused=igb_uio
+ 0000:02:00.0 'NetXtreme BCM5720 Gigabit Ethernet PCIe 165f' if=eth2 drv=tg3 unused=igb_uio
+ 0000:02:00.1 'NetXtreme BCM5720 Gigabit Ethernet PCIe 165f' if=eth3 drv=tg3 unused=igb_uio
+ 0000:04:00.0 'Ethernet Controller 10-Gigabit X540-AT2 1528' if=dpdk0 drv=ixgbe unused=igb_uio *Active*
+ 0000:04:00.1 'Ethernet Controller 10-Gigabit X540-AT2 1528' if=dpdk0 drv=ixgbe unused=igb_uio *Active*
+ 0000:05:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=jenkins1 drv=ixgbe unused=igb_uio *Active*
+ 0000:05:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' if=jenkins1 drv=ixgbe unused=igb_uio *Active*
+ [...]
+
+Here, we have two NICs of two ports each, the **X540** and the **82599ES**. Their identification are their
+PCI bus addresses. To bind the four ports to DPDK::
+
+ $> dpdk-devbind -b igb_uio 04:00.0 04:00.1 05:00.0 05:00.1
+
+Check that the ports have been well binded::
+
+ $> dpdk-devbind.py --status
+
+ [...]
+ Network devices using DPDK-compatible driver
+ ============================================
+ 0000:04:00.0 'Ethernet Controller 10-Gigabit X540-AT2 1528' drv=igb_uio unused=ixgbe
+ 0000:04:00.1 'Ethernet Controller 10-Gigabit X540-AT2 1528' drv=igb_uio unused=ixgbe
+ 0000:05:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' drv=igb_uio unused=ixgbe
+ 0000:05:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb' drv=igb_uio unused=ixgbe
+
+ Network devices using kernel driver
+ ===================================
+ 0000:01:00.0 'NetXtreme BCM5720 Gigabit Ethernet PCIe 165f' if=eth0 drv=tg3 unused=igb_uio *Active*
+ 0000:01:00.1 'NetXtreme BCM5720 Gigabit Ethernet PCIe 165f' if=eth1 drv=tg3 unused=igb_uio
+ 0000:02:00.0 'NetXtreme BCM5720 Gigabit Ethernet PCIe 165f' if=eth2 drv=tg3 unused=igb_uio
+ 0000:02:00.1 'NetXtreme BCM5720 Gigabit Ethernet PCIe 165f' if=eth3 drv=tg3 unused=igb_uio
+ [...]
+
+
+For more detailed explainations on this point, please refer to the related
+`DPDK documentation <https://doc.dpdk.org/guides-18.11/linux_gsg/linux_drivers.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules>`_.
+
+Launching it
+------------
+
+Tool usage::
+
+ dpdk-replay [OPTIONS] PCAP_FILE PORT1[,PORTX...]
+ PCAP_FILE: the file to send through the DPDK ports.
+ PORT1[,PORTX...] : specify the list of ports to be used (pci addresses).
+ Options:
+ --numacore <NUMA-CORE> : use cores from the desired NUMA. Only
+ NICs on the selected numa core will be available (default is 0).
+ --nbruns <1-N> : set the wanted number of replay (1 by default).
+ --wait-enter: will wait until you press ENTER to start the replay (asked
+ once all the initialization are done).
+
+Basic example to replay **foobar.pcap** on the **04:00.0** DPDK port::
+
+ $> dpdk-replay foobar.pcap 04:00.0
+
+
+Replay a **thousand** times the **foobar.pcap** file on **four** DPDK ports.::
+
+ $> dpdk-replay --nbruns 1000 foobar.pcap 04:00.0,04:00.1,04:00.2,04:00.3
+
+
+Use NIC ports from **NUMA 1**::
+
+ $> dpdk-replay --numacore 1 foobar.pcap 81:00.0
+
+Let dpdk-replay prepare the replay and decide when to make it starts by **pressing ENTER**::
+
+ $> dpdk-replay --wait-enter foobar.pcap 04:00.0
diff --git a/docs/src/readme.rst b/docs/src/readme.rst
deleted file mode 100644
index 0f94d06..0000000
--- a/docs/src/readme.rst
+++ /dev/null
@@ -1,68 +0,0 @@
-.. dpdk-burst-replay: BSD-3-Clause
- Copyright 2018 Jonathan Ribas, FraudBuster. All rights reserved.
-
-.. _readme:
-
-README
-======
-
-Introduction
-------------
-
-The tool is designed to provide high DPDK performances to burst any pcap dump on
-a single or multiple NIC port(s).
-
-To do so, the pcap files will be cached on hugepages before being sent through DPDK.
-
-How to play with it
--------------------
-
-Install dependencies
-^^^^^^^^^^^^^^^^^^^^
-
-* dpdk-dev (obsly)
-* libnuma-dev
-* That's all.
-
-NB: libpcap is not required, as dpdk-replay process pcap files manually.
-
-Compiling and installing it
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-::
-
- autoreconf -i && ./configure [--enable-debug] && make && sudo make install
-
-OR::
-
- RTE_SDK=<RTE_SDK_PATH> make -f DPDK_Makefile && sudo cp build/dpdk-replay /usr/bin
-
-Launching it
-^^^^^^^^^^^^
-
-::
-
- dpdk-replay [--nbruns NB] [--numacore 0|1] FILE NIC_ADDR[,NIC_ADDR...]
-
-Example::
-
- dpdk-replay --nbruns 1000 --numacore 0 foobar.pcap 04:00.0,04:00.1,04:00.2,04:00.3
-
-TODO
-----
-
-* Add a configuration file or cmdline options for all code defines.
-* Add an option to configure maximum bitrate.
-* Add an option to send the pcap with the good pcap timers.
-* Add an option to send the pcap with a multiplicative speed (like, ten times the normal speed).
-* Add an option to select multiple pcap files at once.
-* Be able to send dumps simultaneously on both numacores.
-* Split big pkts into multiple mbufs.
-* Add a Python module to facilitate scripting (something like what does scapy for tcpreplay sendpfast func).
-* Manage systems with more than 2 numa cores.
-* Use the maximum NICs capabilities (Tx queues/descriptors).
-
-BSD LICENCE
------------
-
-Copyright 2018 Jonathan Ribas, FraudBuster. All rights reserved.
diff --git a/docs/src/todo-list.rst b/docs/src/todo-list.rst
new file mode 100644
index 0000000..88d4278
--- /dev/null
+++ b/docs/src/todo-list.rst
@@ -0,0 +1,18 @@
+.. dpdk-burst-replay: BSD-3-Clause
+ Copyright 2018 Jonathan Ribas, FraudBuster. All rights reserved.
+
+.. _todo-list:
+
+TODO list
+=========
+
+* Add a configuration file or cmdline options for all code defines.
+* Add an option to configure maximum bitrate.
+* Add an option to send the pcap with the good pcap timers.
+* Add an option to send the pcap with a multiplicative speed (like, ten times the normal speed).
+* Add an option to select multiple pcap files at once.
+* Be able to send dumps simultaneously on both numacores.
+* Split big pkts into multiple mbufs.
+* Add a Python module to facilitate scripting (something like what does scapy for tcpreplay sendpfast func).
+* Manage systems with more than 2 numa cores.
+* Use the maximum NICs capabilities (Tx queues/descriptors).
diff --git a/docs/src/user-guide.rst b/docs/src/user-guide.rst
new file mode 100644
index 0000000..3dbf9df
--- /dev/null
+++ b/docs/src/user-guide.rst
@@ -0,0 +1,104 @@
+.. dpdk-burst-replay: BSD-3-Clause
+ Copyright 2018 Jonathan Ribas, FraudBuster. All rights reserved.
+
+.. _user-guide:
+
+User guide
+==========
+
+.. _installing_custom_dpdk_version:
+
+Installing custom DPDK version
+------------------------------
+
+You should install a custom DPDK version if:
+
+* Your distribution does not propose any DPDK package (obsly).
+* You want to use a different version of what is proposed in your distro.
+* Your NICs are not supported by your distro package (which can be the case for vendors
+ like Mellanox/Broadcom etc).
+* Or, you want to tune :ref:DPDK options.
+
+First, note that dpdk-burst-replay is compatible (ie, checked) with DPDK LTS versions.
+Please refer to :ref:`dpdk_compatible_versions` section.
+
+Please also note that this doc section is not intended to replace the
+`official DPDK compiling guide <https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html>`_,
+but is more focused on giving additional tips related to dpdk-burst-replay purposes.
+
+Get DPDK
+^^^^^^^^
+
+Once you have choose the version you want to use, get its tarball from `download section <https://core.dpdk.org/download/>`_ or directly from `GIT repository <https://git.dpdk.org/dpdk-stable/>`_.
+
+For example::
+
+ $> wget https://fast.dpdk.org/rel/dpdk-18.11.1.tar.xz && tar xvf dpdk-18.11.1.tar.xz
+
+
+.. _customize_dpdk:
+
+Customize DPDK
+^^^^^^^^^^^^^^
+
+DPDK build options are mainly located on **config/common_base** file. Here are some example that could be helpful:
+
+* In lasts versions of DPDK the memory management have changed and there is an hardcoded limit of memory
+ size that can be allocated. And, as dpdk-burst-replay put in cache all pcap packets before sending them
+ it could be an issue if you want to replay big pcap dumps. To be able to use up to 128Go (instead of 32Go
+ by default), set **131272** here::
+
+ CONFIG_RTE_MAX_MEM_MB_PER_LIST=32768
+
+* By default DPDK libs are compiled in static, if you want them to be shared ones, set to 'y' this line::
+
+ CONFIG_RTE_BUILD_SHARED_LIB=n
+
+Build DPDK
+^^^^^^^^^^
+
+Once your configurations have been made, you can follow the
+`official DPDK compiling guide <https://doc.dpdk.org/guides/linux_gsg/build_dpdk.html>`_. For example::
+
+ $> make config T=x86_64-native-linux-gcc
+ $> make -j99 T=x86_64-native-linux-gcc
+
+Install DPDK
+^^^^^^^^^^^^
+
+To install compiled DPDK version into /usr/local, just type::
+
+ $> sudo make install
+
+NOTE: You can change the destination folder by specifying a different **DESTDIR**::
+
+ $> make DESTDIR=~/dpdk-18.11.1 install
+
+
+.. _linking_custom_dpdk_version:
+
+Linking custom DPDK version
+---------------------------
+
+Autotools way
+^^^^^^^^^^^^^
+
+You can specify your custom DPDK version location on **configure** step, for example::
+
+ $> ./configure CFLAGS=-I/home/user/dpdk-18.11.1/usr/local/include/dpdk LDFLAGS=-L/home/user/dpdk-18.11.1/usr/local/lib
+
+Then, compile and install as usual::
+
+ $> make && sudo make install
+
+Static way
+^^^^^^^^^^
+
+Using a static makefile with DPDK's env and conf::
+
+ RTE_SDK=<DPDK_PATH> make -f DPDK_Makefile && sudo cp build/dpdk-replay /usr/bin
+
+Troubleshooting
+---------------
+
+TODO :)