Volume I took the router apart. This volume puts it into service: how the image is built, what runs between power-on and the first forwarded packet, where the Linux kernel sits on the packet path, how the VM and container builds differ, what is actually supported, how to tune it, and how it behaves on virtual NICs and on Compute Engine.
Twelve atoms from NOS to memif, the five Redis databases and seven containers, a BGP route traced from fpmsyncd to the VPP graph, the three packaging form factors, the LCP interface mapping, a five-platform comparison, and the fidelity ladder.
Systems and strategy. The atoms below are only the ones Volume I did not cover. Everything here was checked against source code, not just documentation, because the docs in this project trail the code by a year in places.
These are the parts you meet as soon as you build, boot or tune SONiC-VPP. Tap one for its definition. Later sections use these words without stopping to define them again.
Five repositories produce SONiC-VPP. Only two contain code written for it: sonic-platform-vpp, which holds the build rules, the VPP patches and plugins, the containers and the boot scripts, and sonic-sairedis/vslib/vpp, which translates SAI calls into VPP API messages. Everything else is stock SONiC or stock VPP.
vpp_init.sh under supervisord. Owns the data NICs, the forwarding tables and the LCP taps.platform-modules-vpp package. Runs on first boot, discovers the data ports and writes the port plan.vpp-port-config.service before syncd starts. Moves each data NIC from its kernel driver to a userspace I/O driver, and back on stop.PLATFORM|vpp entry in CONFIG_DB when VPP_CONF_DB=y. Packaged in the single container.Every VPP change SONiC carries, from vppbld/patches/series and vppbld/plugins/src. Gaps in numbering are retired patches.
+b1sonic1.vxlan_main so other plugins can reach VXLAN state.SONIC_EXT_CONFIG.Only a third of the carried code adds features. The rest changes VPP defaults that a switch ASIC does not have: VPP answers IPv6 neighbor discovery itself, sends its own ICMP errors, resolves next hops itself and drops some addresses on principle. SONiC expects the hardware to trap those packets to the CPU and let Linux decide. Much of SONiC-VPP's engineering goes into making a software router act like silicon with a CPU port.
SONiC-VPP is a platform inside SONiC's normal build, the way a switch vendor is. You configure the build for PLATFORM=vpp, and the build pulls in platform/vpp, a git submodule pinned to a commit of sonic-platform-vpp.
make init checks out all submodules, including platform/vpp at the commit sonic-buildimage pins.make configure PLATFORM=vpp builds the Debian trixie (and bookworm) sonic-slave images. This is most of the first-build time; later builds reuse them.VPP_VERSION_SONIC from the sonic-vpp Buildkite registry. On a miss it clones gerrit.fd.io, checks out the SHA in vppbld/vpp_version, copies in the three plugins, applies the 19 patches and runs VPP's own make pkg-deb with the Mellanox PMDs turned off.psample for sFlow and mpls_router for MPLS at boot.git clone --recurse-submodules https://github.com/sonic-net/sonic-buildimage.git cd sonic-buildimage make init NOBULLSEYE=1 NOBUSTER=1 make configure PLATFORM=vpp NOBULLSEYE=1 NOBUSTER=1 make SONIC_BUILD_JOBS=4 target/sonic-vpp.img.gz # output: target/sonic-vpp.img.gz (CI also builds INSTALL_DEBUG_TOOLS=y)
sudo modprobe ip_tables iptable_nat iptable_filter # needed by docker-in-slave NOBULLSEYE=1 NOBUSTER=1 BUILD_SKIP_TEST=y INCLUDE_FIPS=n \ make SONIC_BUILD_JOBS=4 target/docker-sonic-vpp.gz
Full multi-container SONiC, installed through an ONIE recovery ISO inside QEMU during the build. This is what sonic-mgmt tests every night.
The same image for x86 servers that boot ONIE. Not exercised by the public nightly runs.
The same image as a raw disk, for platforms that import raw disks rather than qcow2.
A reduced SONiC in one container: no SNMP, LLDP, telemetry or platform monitoring. It needs INCLUDE_FIPS=n and has no sanity tests in CI.
VPP_VERSION in rules/vpp.mk (today 2606-0.10) names the packages, and the build fetches prebuilt packages by that name. Change a patch or a plugin without bumping the minor number and every downstream build silently downloads the old packages, while libsaivs compiles against your new headers. The result is a VPP and a SAI library that disagree about message CRCs. The rule in vpp.mk is to bump on any change under vppbld/.
Also: the packages are labelled 2606, but the pinned commit is from the 26.10 development cycle on VPP master. The label tracks the base release; the SHA is what actually ships.
On the KVM image, eight things happen in order between the hypervisor handing over NICs and VPP forwarding. Most problems people hit, such as ports missing, VPP not starting, or the second port being named Ethernet4, trace back to one of these steps.
syncd_vpp_env, the ifmap and the initial config_db.json come from a factory-default hook. They are not regenerated on reboot. If you add, remove or reorder data NICs after first boot, VPP will be started with a stale PCI list. Regenerate by re-running the hook (or config-setup factory) rather than editing the environment of the syncd container, which reads the file through its /etc/sonic mount.
VPP forwards transit traffic. The Linux kernel never does. But the kernel still owns everything SONiC's daemons need it to own: the neighbor tables, the sockets for BGP, LACP through teamd, ICMP errors, and the routes FRR installs. Every packet that needs one of those crosses a TAP. Pick a packet and watch where it goes.
zebra still installs every route into the kernel, so ip route inside SONiC looks complete. Nothing on the transit path reads it. VPP's linux-nl plugin, which would mirror kernel changes into VPP, is turned off (NO_LINUX_NL=y). The only way state reaches VPP is fpmsyncd → APPL_DB → orchagent → ASIC_DB → SwitchVpp. The kernel FIB is used only for packets the router itself originates, which leave through a TAP and are cross-connected to the wire.
bobm0, created from startup.conf, paired with TAP Ethernet0. Direct pair.BondEthernet10 paired with helper TAP be10. A tc filter redirects be10's ingress to the kernel's PortChannel10, where teamd runs LACP. sonic_ext sends LACP frames up the member's own TAP so teamd knows the arrival port.loop0 paired with tap_Loopback0, tc-redirected to Loopback0.bobm0.10 and Ethernet0.10 are created automatically under the parent pair (lcp-auto-subint).vhost-net, so punts and host-originated packets cost a vhost round trip.uio_pci_generic or vfio-pci hold the data NICs. Kernel networking never sees them again until syncd stops.hsflowd over the kernel's psample netlink channel, which the image loads at boot.mpls_router so intfmgrd can enable MPLS on the TAPs; the labels themselves are programmed into VPP.Volume I compared the packaging options at a glance. The difference that matters in practice is where the kernel is and how ports get in. A VM has its own kernel and discovers its NICs at boot. The container borrows the host's kernel and is handed its ports by a launch script.
Status here is graded on evidence in code and tests, not roadmap. "Supported" means the SAI translation exists and sonic-mgmt or plugin tests exercise it. "Partial" means code exists but coverage is thin or explicitly skipped. The project README's own feature list dates from the first phase and is years behind the code.
VPP's speed comes from three things: forwarding threads pinned to dedicated cores, each NIC queue polled by exactly one thread, and packet buffers in hugepages. SONiC-VPP ships tuned for small lab VMs, not for throughput. Its defaults are one worker, one receive queue per port, 2,048 buffers per port and a couple dozen megabytes of 2 MB hugepages. Every one of those has to change before the numbers look like VPP's published ones.
A worker spins at 100% of a core whether or not traffic flows. In September 2026 the project cut the VM default from two workers to one after measuring a two-DUT KVM testbed: VPP CPU per DUT fell from 282% to 178%, host CPU for both DUTs fell by about a third, and a flaky BGP test began passing because the control plane got its cores back. The virtio PMD used in KVM testbeds cannot run in interrupt or adaptive mode under VPP, so worker count is the only lever.
Reference points, not promises: on Compute Engine one VPP worker forwarded about 18 Mpps of 64-byte IPv4 on a c3-highcpu-4, and 19 workers reached about 108 Mpps on an h3-standard-88 (roughly 5.7 Mpps per worker once scaled). A single flow always lands on one queue, so one worker is the ceiling for any one 5-tuple.
docker exec -it syncd vppctl show runtime # vectors/call near 256 = thread saturated; near 1 = idle docker exec -it syncd vppctl show hardware-interfaces # rx missed / no-buf = rings or buffers starved docker exec -it syncd vppctl show interface rx-placement # which thread polls which queue docker exec -it syncd vppctl show buffers # pool size and free count docker exec -it syncd vppctl show errors # per-node drop reasons, CoPP policer drops included docker exec -it syncd vppctl show threads # main and worker lcores actually used
The SONiC-VPP scripts treat every port as one of two things: a veth, which VPP reads through AF_PACKET, or a PCI device, which DPDK drives. What sits behind that PCI device decides everything about performance. Pick a NIC type and compare the path a packet takes from a VPP worker to the wire.
Compute Engine's third-generation machines (C3, C3D, H3, C4 and later) offer only gVNIC, implemented on the Titanium offload hardware rather than in host software. VPP's bundled DPDK includes the gve driver, so VPP can drive a gVNIC directly. The network around the VM is the harder part: a VPC is a routed L3 network with no shared Ethernet segment, so anything SONiC assumes about wires has to be rethought.
--can-ip-forward. Point VPC routes at the VM (next hop instance or IP), or peer BGP with Cloud Router through an NCC Router Appliance.SONiC-VPP is developed in the open under the SONiC Virtual Dataplane Workgroup, chaired by Cisco engineers. In the twelve months to October 2026, sonic-platform-vpp took 79 commits, most from Microsoft and Cisco, with contributions from PANTHEON.tech and Nexthop. A dataplane feature usually touches three repositories in this order.
docs/HLD of sonic-platform-vpp: mirror and Everflow, IP-in-IP, route flow counters and LAG all started this way.vlib_get_plugin_symbol, so stock VPP behaves exactly as before when the plugin is absent.make ci-build; make ci-test runs VPP's own unit tests plus the plugins' tests. A preflight fails the job if network namespaces can't be created, because those suites would otherwise skip silently and pass.sonic-sairedis/vslib/vpp, with unit tests there.platform/vpp submodule, so the KVM image goes through sonic-mgmt sanity before the platform PR merges.tests_mark_conditions_sonic_vpp.yaml, which currently lists 61 skips, most marked "to be included".Read the commit log and a pattern shows. Recent work was driven by sonic-mgmt test cases: drop counters on t0, CoPP on t1-lag, BGP on dual-ToR, Everflow. The KVM image is tested nightly; the single container is not tested at all; and a commit keeping the syncd-vpp container privileged carries the note that it is not used in production. Cisco's NANOG 95 talk also reports cloud providers using it as a gateway, but the engineering center of gravity is test fidelity.
VPP has been fast for a decade. What SONiC-VPP spends its effort on is making VPP's behavior match what SONiC's tests expect from an ASIC: who answers ND, who sends ICMP errors, which ingress port a punted LACP frame reports, how a CoPP policer counts. Each shim is a place where VPP and real silicon can disagree. If you rely on SONiC-VPP to predict hardware behavior, the sonic_ext feature list is the list of things to double-check.
Queueing, buffering, PFC or WRED behavior; warm-reboot testing; anything that depends on a specific ASIC's pipeline quirks; and multi-terabit throughput. Use hardware for those.
Volume III could open the translation layer itself: SwitchVpp's object model, how it fakes single-attribute SAI updates on an API that has none, what bulk calls cost, and how CRC drift between VPP and libsaivs shows up at runtime.