OVERLAY(7) Standards, Environments, and Macros OVERLAY(7)

NAME


overlay - Overlay Devices

DESCRIPTION


Overlay devices are a GLDv3 device that allows users to create overlay
networks that can be used to form the basis of network virtualization
and software defined networking. Overlay networks allow a single
physical network, often called an underlay network, to provide the
means for creating multiple logical, isolated, and discrete layer two
and layer three networks on top of it.

Overlay devices are administered through dladm(8). Overlay devices
themselves cannot be plumbed up with IP, vnd, or any other protocol.
Instead, like an etherstub, they allow for VNICs to be created on top
of them. Like an etherstub, an overlay device acts as a local switch;
however, when it encounters a non-local destination address, it instead
looks up where it should send the packet, encapsulates it, and sends it
out another interface in the system.

A single overlay device encapsulates the logic to answer two different,
but related, questions:

1. How should a packet be transformed and put on the wire?
2. Where should a transformed packet be sent?

Each of these questions is answered by a plugin. The first question is
answered by what's called an encapsulation plugin. The second question
is answered by what's called a search plugin. Packets are encapsulated
and decapsulated using the encapsulation plugin by the kernel. The
search plugins are all user land plugins that are consumed by the varpd
service whose FMRI is svc:/network/varpd:default. This separation
allows for the kernel to be responsible for the data path, while having
the search plugins in userland allows the system to provide a much more
expressive interface.

Overlay Types


Overlay devices come in two different flavors, one where all packets
are always sent to a single address, the other, where the destination
of a packet varies based on the target MAC address of the packet. This
information is maintained in a target table, which is independent and
unique to each overlay device. We call the plugins that send traffic
to a single location, for example a single unicast or multicast IP
address, a point to point overlay and the overlay devices that can send
traffic to different locations based on the MAC address of that packet
a dynamic overlay. The plugin type is determined based on the type of
the search plugin. These are all fully listed in the section Plugins
and their Properties.

Overlay Destination


Both encapsulation and search plugins define the kinds of destinations
that they know how to support. An encapsulation plugin always has a
single destination type that's determined based on how the
encapsulation is defined. A search plugin, on the other hand, can
support multiple combinations of destinations. A search plugin must
support the destination type of the encapsulation device. The
destination may require any of the following three pieces of
information, depending on the encapsulation plugin:

MAC Address
An Ethernet MAC address is required to determine the
destination.

IP Address
An IP address is required. Both IPv4 and IPv6 addresses are
supported.

Port
An IP protocol level (TCP, UDP, SCTP, etc.) port is required.

The list of destination types that are supported by both the search and
encapsulation plugins is listed in the section Plugins and their
Properties.

varpd
The varpd service, mentioned above, is responsible for providing the
virtual ARP daemon. Its responsibility is conceptually similar to ARP.
It runs all instances of search plugins in the system and is
responsible for answering the kernel's ARP-like questions for where
packets should be sent.

The varpd service, svc:/network/varpd:default, must be enabled for
overlay devices to function. If it is disabled while there are active
devices, then most overlay devices will not function correctly and
likely will end up dropping traffic.

PLUGINS AND PROPERTIES


Properties fall into three categories in the system:
1. Generic properties all overlay devices have
2. Properties specific to the encapsulation plugin
3. Properties specific to the search plugin

Each property in the system has the following attributes, which mirror
the traditional dladm(8) link properties:

Name
The name of a property is namespaced by its module and always
structured and referred to as as module/property. This allows
for both an encapsulation and search plugin to have a property
with the same name. Properties that are valid for all overlay
devices and not specific to a module do not generally use a
module prefix.

For example, the property vxlan/listen_ip is associated with
the vxlan encapsulation module.

Type
Each property in the system has a type. dladm(8) takes care of
converting between the internal representation and a value, but
the type influences the acceptable input range. The types are:

INT A signed integer that is up to eight bytes long
(int64_t).

UINT An unsigned integer that is up to eight bytes long
(uint64_t).

IP Either an IPv4 or IPv6 address in traditional string
form. For example, 192.168.128.23 or
2001:470:8af4::1:1. IPv4 addresses may also be encoded
as IPv4-mapped IPv6 addresses.

STRING A string of ASCII or UTF-8 encoded characters
terminated with a NUL byte. The maximum string length,
including the terminator, is currently 256 bytes.

Permissions
Each property has permissions associated with it, which
indicate whether the system considers them read-only properties
or read-write properties. A read-only property can never be
updated once the device is created. This generally includes
things like the overlay's encapsulation module.

Required
This property indicates whether the property is required for
the given plugin. If it is not specified during a call to
dladm create-overlay, then the overlay cannot be successfully
created. Properties which have a default will use that value
if one is not specified rather than cause the overlay creation
to fail.

Current Value
The current value of a property, if the property has a value
set. Required properties always have a value set.

Default Value
The default value is an optional part of a given property. If
a property does define a default value, then it will be used
when an overlay is created and no other value is given.

Value ranges
Value ranges are an optional part of a given property. They
indicate a range or set of values that are valid and may be set
for a property. A property may not declare such a range as it
may be impractical or unknown. For example, most properties
based on IP addresses will not declare a range.

The following sections describe both the modules and the properties
that exist for each module, noting their name, type, permissions,
whether or not they are required, and if there is a default value. In
addition, the effects of each property will be described.

Encapsulation Plugins


vxlan The vxlan module is a UDP based encapsulation method. It takes
a frame that would be put on the wire, wraps it up in a VXLAN
header and places it in a UDP packet that gets sent out on the
underlying network. For more details about the specific format
of the VXLAN header, see vxlan(4P).

The vxlan module requires both an IP address and port to
address it. It has a 24-bit virtual network ID space, allowing
for virtual network identifiers that range from 0 - 16777215.

The vxlan module has the following properties:

vxlan/listen_ip
Type: IP | Permissions: Read/Write | Required

The vxlan/listen_ip property determines the IP address
that the system will accept VXLAN encapsulated packets
on for this overlay.

vxlan/listen_port
Type: UINT | Permissions: Read/Write | Required
Default Value: 4789 | Range: 0 - 65535

The vxlan/listen_port property determines the UDP port
that the system will listen on for VXLAN traffic for
this overlay. The default value is 4789, the IANA
assigned port for VXLAN.

The vxlan/listen_ip and vxlan/listen_port properties determine
how the system will accept VXLAN encapsulated packets for this
interface. It does not determine the interface that packets
will be sent out over. Multiple overlays that all use VXLAN
can share the same IP and port combination, as the virtual
network identifier can be used to tell the different overlays
apart.

Search Plugins


Because search plugins may support multiple destinations, they may have
more properties listed than necessarily show up for a given overlay.
For example, the direct plugin supports destinations that are
identified by both an IP address and a port, or just an IP address. In
cases where the device is created over an overlay that only uses an IP
address for its destination, then it will not have the direct/dest_port
property.

direct The direct plugin is a point to point module that can be used
to create an overlay that forwards all non-local traffic to a
single destination. It supports destinations that are a
combination of an IP Address and a port.

The direct plugin has the following properties:

direct/dest_ip
Type: IP | Permissions: Read/Write | Required

The direct/dest_ip property indicates the IP address
that all traffic will be sent out. Traffic will be
sent out the corresponding interface based on
traditional IP routing rules and the configuration of
the networking stack of the global zone.

direct/dest_port
Type: UINT | Permissions: Read/Write | Required
Default Value: - | Range: 0 - 65535

The direct/dest_port property indicates the TCP or UDP
port that all traffic will be directed to.

files The files plugin implements a dynamic plugin that specifies
where traffic should be sent based on a file. It is a
glorified version of /etc/ethers. The dynamic plugin does not
support broadcast or multicast traffic, but it has support for
proxy ARP, NDP, and DHCPv4. For the full details of the file
format, see overlay_files(5).

The files plugin has the following property:

files/config
Type: String | Permissions: Read/Write | Required

The files/config property specifies an absolute path to
a file to read. The file is a JSON file that is
formatted according to overlay_files(5).

General Properties


Each overlay has the following properties which are used to give
additional information about the system. None of these properties may
be specified as part of a dladm create-overlay, instead they come from
other arguments or from internal parts of the system.

encap
String | Permissions: Read Only

The encap property contains the name of the encapsulation
module that's in use.

mtu
UINT | Permissions: Read/Write
Default Value: 1400 | Range: 576 - 9000

The mtu property describes the maximum transmission unit of the
overlay. The default value is 1400 bytes, which ensures that
in a traditional deployment with an MTU of 1500 bytes, the
overhead that is added from encapsulation is all accounted for.
It is the administrator's responsibility to ensure that the
device's MTU and the encapsulation overhead does not exceed
that of the interfaces that the encapsulated traffic will be
sent out of.

To modify the mtu property, use dladm set-linkprop.

search
String | Permissions: Read Only

The search property contains the name of the search plugin
that's in use.

varpd/id
String | Permissions: Read Only

The varpd/id property indicates the identifier which the varpd
service uses for this overlay.

vnetid
UINT | Permissions: Read/Write

The vnetid property has the virtual network identifier that
belongs to this overlay. The valid range for the virtual
network identifier depends on the encapsulation engine.

FMA INTEGRATION


Overlay devices are wired into FMA, the illumos fault management
architecture, and generates error reports depending on the search
plugin in use. Due to limitations in FMA today, when a single overlay
enters a degraded state, meaning that it cannot properly perform look
ups or another error occurred, then it degrades the overall overlay
pseudo-device driver.

For more fine-grained information about which overlay is actually in a
degraded state, one should run dladm show-overlay -f. In addition, for
each overlay in a degraded state a more useful diagnostic message is
provided which describes the reason that caused this overlay to enter
into a degraded state.

The overlay driver is self-healing. If the problem corrects itself on
its own, it will clear the fault on the corresponding device.

SEE ALSO


vxlan(4P), overlay_files(5), dladm(8)

illumos March 30, 2022 illumos

tribblix@gmail.com :: GitHub :: Privacy