Web and networkingDive 08 of 12
WebRTC, from Join call to the wire
Ana is behind her home router. Ben is behind a carrier NAT. Follow their video call through negotiation, hole punching, encryption and the packets that arrive just in time.
- Levels
- 10
- Reading
- 50 min
- Depth
- Idea to Machine
- Published
Ana and Ben can both send out. Can a packet get back in?
- Selected path
- TURN relay
- One-way network delay
- 80 ms
- IP bytes across public legs / video packet
- 2,106 B
Ben sends from a different public port than Ana contacted. Her router filters it. A TURN relay supplies a reachable address for this call.
Computed with the NAT lab’s model, not the network name alone. Ana uses endpoint-independent mapping and address-and-port filtering; Ben uses that home setup or address-and-port-dependent mapping/filtering. Illustrative delays: local links 5 + 10 ms, direct public leg 25 ms, relay legs 35 + 30 ms. A 1,051-byte IPv4 packet gains 4 ChannelData bytes on Ben’s TURN leg.
Two browsers, no listening socket
Ana presses Join call
on her laptop. Her home router has one public IPv4 address shared by every device in the house. Ben answers on his phone, whose mobile carrier shares public addresses across many subscribers. Neither browser has a public listening socket. A packet addressed to Ben’s phone
has nowhere obvious to go.
Yet a moment later Ana sees Ben wave. Her microphone produces a steady stream of short audio frames; her camera produces much larger video frames. A chat panel carries the occasional Can you hear me?
The three streams share a connection, but they have different priorities. A chat sentence should eventually arrive intact. Yesterday’s missing syllable is useless once the conversation has moved on.
You could carry everything through your application server. It already knows Ana and Ben, and both can make outbound HTTPS connections to it. That buys a simple rendezvous point at the cost of moving every media byte twice: into the server and back out again. At 1.5 Mbit/s per sender, a two-person call needs 3 Mbit/s of server egress before headers, retransmissions or redundancy. A thousand such calls need roughly 3 Gbit/s. Geographic detours can also add delay, although a well-placed relay can sometimes beat a poor direct route.
Audio turns delay into friction. Ana finishes a sentence, Ben waits to be sure she has stopped, and both begin speaking. Capturing sound, encoding it, waiting in queues, crossing the network, buffering jitter and driving a speaker each consume part of the budget. Saving a server hop helps, but it cannot repair a congested home uplink or a headset with a large device buffer.
Home Wi-Fi
Mobile carrier
WebRTC is the browser’s answer to this whole problem, rather than just a socket API. It combines media capture and playback APIs with codec negotiation, connectivity checks, encryption, congestion control and packet scheduling. The browser can encode the camera, discover a route, protect the packets and reconstruct sound despite loss. Your JavaScript describes the desired connection and reacts to its state.
One conspicuous piece is yours: The application-defined exchange of session descriptions, ICE candidates and call-control messages before and during a call.. WebRTC does not decide which person to ring, how a room works, how users authenticate or how an offer reaches another browser. An authenticated WebSocket is a common transport; an HTTP exchange or existing messaging system can work too. The protocol carrying the invitation does not need to carry the eventual video.
| Choice | What it buys | What you still handle |
|---|---|---|
| WebSocket through your server | Reliable, ordered application messages | Media codecs, playout deadlines, transport adaptation |
| WebRTC with direct ICE pair | Browser media engine and peer transport | Signaling, identity, reachable fallback |
| WebRTC through TURN | Same peer security across a relay path | Relay credentials, capacity and availability |
| WebRTC to a media server | Group routing, recording or mixing | Server media architecture and its trust boundary |
WebRTC moves the time-sensitive transport into the browser. Your application still decides who is on the other end.
Agree before you connect
Ana’s click starts two separate activities: obtaining permission to capture devices and preparing a connection. getUserMedia() returns tracks, one for audio and one for video. An RTCPeerConnection owns the transport machinery. Adding a track associates a sender with that connection; it does not immediately send usable media to Ben. The browsers first need to agree on what each side can send and receive.
The agreement uses offer/answer. Ana creates an offer describing media sections, supported codecs, ICE credentials and a certificate fingerprint. She applies it locally, then sends it through the application’s signaling channel. Ben applies it as a remote description, creates and applies an answer, then sends that answer back. Ana applies the answer remotely. Each side now has the same negotiated session, viewed from opposite directions.
The explicit API sequence is createOffer(), setLocalDescription(offer), delivery, then setRemoteDescription(offer) at Ben. Ben uses createAnswer() and the corresponding setters. Creating a description only produces a proposal. Applying it changes the connection’s state and configures the media engine. The argument-free setLocalDescription() can create the appropriate offer or answer as part of applying it.
JSEP, RFC 9429, which obsoletes RFC 8829, describes this division between application-controlled signaling and browser-controlled media. A sender and receiver are paired in an RTCRtpTransceiver; a transceiver corresponds to an audio or video media section. Its negotiated direction can be send/receive, send-only, receive-only or inactive. Muting a microphone track and changing the negotiated direction are different operations.
Device permission can fail, and successful capture does not guarantee autoplay. Install signaling handlers before adding tracks. Call capture from the join action, mute local preview, and offer an explicit playback button if remote audio cannot autoplay. A rejected promise must leave the interface able to retry. The sample uses application-owned joining, status and media elements; it runs only when installed in a real call application.
// pc and the ordered signaling handlers below are already installed.
// localVideo is muted and both video elements use playsinline.
let joining = false;
let joined = false;
if (isAna) attachChat(pc.createDataChannel("chat"));
joinButton.addEventListener("click", async () => {
if (joining || joined) return;
joining = true;
joinButton.setAttribute("aria-disabled", "true");
let stream;
const added = [];
try {
stream = await navigator.mediaDevices.getUserMedia({
audio: true,
video: true,
});
localVideo.srcObject = stream;
await localVideo.play();
for (const track of stream.getTracks()) {
added.push(pc.addTrack(track, stream));
}
joined = true;
status.textContent = "Joining…";
} catch (error) {
for (const sender of added) pc.removeTrack(sender);
stream?.getTracks().forEach((track) => track.stop());
status.textContent = "Could not join. Check device access and retry.";
} finally {
joining = false;
joinButton.setAttribute("aria-disabled", String(joined));
}
});
pc.ontrack = ({ streams }) => {
remoteVideo.srcObject = streams[0];
remoteVideo.play().catch(() => {
status.textContent = "Press Play remote audio to hear Ben.";
});
};
playRemoteButton.addEventListener("click", () => {
// Invoke play() in the click handler; activation may not survive await.
remoteVideo.play().catch(() => {
status.textContent = "Playback failed. Check output and retry.";
});
});What if both browsers offer?
Ben switches cameras just as Ana adds screen sharing. Both get negotiationneeded, both make an offer, and both enter have-local-offer. That collision is called Two peers independently make offers before receiving the other's offer.. Without a rule, each waits for an answer to a proposal that the other cannot accept in its current state.
The perfect negotiation pattern assigns one peer the polite role. Ben is polite; he abandons his colliding proposal through rollback and accepts Ana’s. Ana is impolite; she ignores Ben’s colliding offer and the candidates belonging to it. Politeness does not mean always answering or always being the called party. When Ana is stable and Ben offers normally, Ana answers normally.
Make both sides speak at once
Ana · impolite
stable
Ben · polite
stable
Choose an offer, then exchange descriptions.
Serialized operations; descriptions are delivered in order per sender. The model includes implicit rollback and ignored-offer candidates, without opening a connection.
The state check needs more than a quick read of signalingState: creating and applying descriptions are asynchronous. A makingOffer flag covers the interval before the visible state changes. The pending-answer flag accounts for an answer being applied while another offer arrives. Modern setRemoteDescription(offer) performs implicit rollback where required; you do not need to race an explicit rollback against a separate remote-description operation.
// signal is your authenticated, reliable, ordered signaling transport.
// send() enqueues a JSON message; no media bytes go through it.
const pc = new RTCPeerConnection({ iceServers: deploymentIceServers });
const polite = !isAna;
let makingOffer = false;
let ignoreOffer = false;
let isSettingRemoteAnswerPending = false;
pc.onnegotiationneeded = async () => {
try {
makingOffer = true;
await pc.setLocalDescription(); // offer in stable; answer for an offer
signal.send({ description: pc.localDescription });
} catch (error) {
reportCallError(error); // announce failure and expose reconnect/retry
} finally {
makingOffer = false;
}
};
pc.onicecandidate = ({ candidate }) => {
// Includes empty candidate strings (end of a generation).
if (candidate !== null) {
try {
signal.send({ candidate });
} catch (error) {
reportCallError(error);
}
}
};
// Serialize received messages: descriptions precede their candidates.
let incoming = Promise.resolve();
signal.onmessage = ({ description, candidate }) => {
incoming = incoming.then(async () => {
if (description) {
const readyForOffer = !makingOffer &&
(pc.signalingState === "stable" || isSettingRemoteAnswerPending);
const collision = description.type === "offer" && !readyForOffer;
ignoreOffer = !polite && collision;
if (ignoreOffer) return;
isSettingRemoteAnswerPending = description.type === "answer";
try {
// A colliding offer implicitly rolls back the polite peer.
await pc.setRemoteDescription(description);
} finally {
isSettingRemoteAnswerPending = false;
}
if (description.type === "offer") {
await pc.setLocalDescription();
signal.send({ description: pc.localDescription });
}
} else if (candidate) {
try {
await pc.addIceCandidate(candidate);
} catch (error) {
if (!ignoreOffer) throw error;
}
}
}).catch(reportCallError);
};The application must preserve signaling order and associate messages with the right call and ICE generation. Candidate messages can arrive while a description is still being applied, so this example serializes incoming operations. A signaling system that can reorder descriptions and candidates needs a queue keyed by generation. Reconnection also needs application state: which room is this, which peer is still present, and which messages are obsolete?
The call, written down
The description’s bulky text is Session Description Protocol: a line-oriented format for describing a multimedia session, not a transport for the media itself.. RFC 8866 defines the base grammar. Each line begins with a one-letter field and =; transmitted lines end with CRLF. Session fields come first, followed by one or more media descriptions. Attributes extend the format without changing that skeleton.
Read Ana’s offer as a set of constraints. The m=audio section offers Opus, the m=video section offers VP8 and H.264, and the m=application section offers SCTP data channels. Ben’s answer selects compatible capabilities. It cannot invent a codec Ana did not offer. Payload type numbers such as 111 and 96 are negotiated labels; they are not universal codec identifiers.
The o= line contains a session ID and revision, so a changed offer is distinguishable from an old one. Its origin address is not an instruction to send media to loopback. Similarly, port 9 and 0.0.0.0 in this initial offer are placeholders used before the actual ICE destination is known. You cannot extract a usable video destination by reading only the m= and c= lines.
Read Ana’s offer, one line at a time
A compact Unified Plan style offer with illustrative credentials and fingerprint; candidates arrive later. Select a line with the mouse or Tab and Enter.
SDP syntax version zero. This is not a WebRTC version.
Several media sections, one transport
a=group:BUNDLE 0 1 2 proposes sharing a transport between the three sections, whose identifiers are declared by a=mid. BUNDLE, RFC 9143, avoids a separate connection setup for each stream. The MID RTP header extension helps associate incoming packets with the right media section. An SSRC identifies an RTP source; it is a different namespace from MID and from the JavaScript track ID.
a=rtcp-mux puts RTP and its control protocol, RTCP, on the same transport. Together with BUNDLE this can leave Ana with one UDP socket carrying audio, video, control traffic and DTLS. The browser distinguishes STUN, DTLS and RTP/RTCP packets before routing them to their individual handlers. Multiplexing does not turn chat into an RTP codec; chat will take the SCTP branch inside DTLS.
The a=rtpmap lines assign codec names and clocks to payload types. Opus uses opus/48000/2 in SDP even when the microphone captures mono speech. The channel count in that capability line does not prove that every encoded frame is stereo. Video’s 90 kHz clock measures presentation time independently of the camera’s frame rate.
a=fmtp carries codec parameters. The H.264 value 42e01f describes a constrained-baseline-compatible profile and level 3.1; packetization-mode=1 permits the non-interleaved packetization mode, including fragmentation units. Opus’s useinbandfec=1 advertises support for receiving in-band forward error correction. It does not promise that the sender will include redundancy in every packet.
Source metadata connects this back to the DOM. msid:ana video0 associates a stream and track identity. A CNAME allows RTP synchronization contexts to be related; RTCP timing reports will map different codec clocks onto a common timebase. Audio and video do not synchronize by having numerically equal RTP timestamps.
Credentials are not media keys
ice-ufrag and ice-pwd authenticate connectivity checks. The SHA-256 fingerprint identifies the certificate expected during DTLS; RFC 8122 defines its syntax. setup:actpass lets the answer choose the DTLS client or server role, following the DTLS-SRTP framework in RFC 5763. Neither attribute contains the eventual SRTP traffic keys.
The data section has its own logical sctp-port and receive message-size limit, defined for this transport in RFC 8841. Port 5000 here is inside DTLS. It does not ask a router to expose UDP port 5000. At every layer, names and ports mean something within that layer’s scope.
SDP describes capabilities, identities and transport parameters. ICE supplies a usable path; DTLS establishes the keys.
Find a path through two NATs
Ana knows what Ben wants to receive. She still does not know where packets can reach him. ICE, RFC 8445, turns possible addresses into tested routes. A One transport address that an ICE agent might use, with type, priority, component, foundation and related metadata. is a possibility; a candidate pair is one local possibility joined to one remote possibility.
A host candidate comes from an interface. Ana might have a private Wi-Fi address, an IPv6 address, and a VPN interface. Private addresses generally cannot reach another household. Browsers may represent local addresses with mDNS names for privacy. A globally reachable IPv6 host candidate can avoid IPv4 address translation, although firewalls still matter.
A server-reflexive candidate, abbreviated srflx, is the public address a STUN server observes when a request leaves Ana’s network. The router translates her internal source address and port into a public endpoint and remembers how to send replies back. A relay candidate is an address allocated on a TURN server. A peer-reflexive candidate can be discovered during checks when a peer observes a source address that was not in the signaled list.
With Trickle ICE, RFC 8838, Ana sends candidates as she discovers them. Ben need not wait for every interface, STUN response and TURN allocation before starting checks. Candidates belong to a media transport and an ICE generation; the username fragment helps identify the generation. End-of-candidates is meaningful information too: a slow candidate is different from a candidate that will never arrive.
A preference is not a ping measurement
The recommended candidate priority packs three preferences into an unsigned integer. Type preference occupies the high bits; local preference distinguishes interfaces or address families; the component favors RTP over a separate RTCP component. With RTCP multiplexing the media uses component 1. The recommended type preferences include host 126, peer-reflexive 110, server-reflexive 100 and relay 0.
The pair combines the controlling agent’s candidate priority G and the controlled agent’s priority D. The minimum dominates: a pair with one low-priority end stays low. The final bit breaks symmetry consistently using the ICE roles. These roles do not define who makes future SDP offers or who becomes the DTLS client.
Compare candidate-pair priorities
Ana’s candidates × Ben’s candidates. Each cell computes the exact 64-bit pair priority.
Ana priority 2130706431; Ben priority 2130706175. Pair priority 9151313343271665663. Ana supplies G (controlling), the other supplies D. host ↔ host still needs a successful STUN check.
Component 1; type preferences: host 126, server-reflexive 100, relay 0. Priority ranks checks; it cannot prove reachability.
ICE makes checklists from compatible candidates and sends authenticated STUN Binding requests on candidate pairs. Successful checks create valid pairs. A controlling agent then nominates a pair, using USE-CANDIDATE in a check, so both ends agree which route will carry the traffic. A high-ranked pair can fail; the agent proceeds to other possibilities. Foundations group candidates with similar underlying paths so checks can be paced rather than flooding every pair at once.
Mapping and filtering are separate decisions
A router makes two decisions. Its mapping behavior decides which public source endpoint to use for an outbound packet. Endpoint-independent mapping reuses the same mapping for a given internal endpoint. Address-dependent mapping changes with the remote IP; address-and-port-dependent mapping can change with both remote IP and port. Its filtering behavior decides which outside sources may send back through an existing mapping.
The terms come from RFC 4787. Endpoint-independent filtering accepts any external source on an existing binding. Address-dependent filtering requires that the inside endpoint has sent to that outside IP. Address-and-port-dependent filtering requires the exact endpoint. Sending checks outward can open filters on both sides, which is the basis of UDP hole punching. There is no special packet that orders a remote router to open a hole.
Change the routers, keep the same browsers
Ana · home Wi-Fi
Ben · carrier NAT
TURN relay needed in this model. Ana creates 1 mapping(s); Ben creates 2. Host ↔ host: unreachable across these private networks. Server-reflexive / peer-reflexive: checks fail. Relay ↔ peer: reachable with the assumed TURN allocation.
- STUN publishes Ana 198.51.100.10:40000 and Ben 203.0.113.20:50000.
- 198.51.100.10:40000 → 203.0.113.20:50000: request accepted.
- 203.0.113.20:50001 → 198.51.100.10:40000: filtered.
- 198.51.100.10:40000 → 203.0.113.20:50000: request accepted.
- 198.51.100.10:40000 → 203.0.113.20:50000: request accepted.
- 198.51.100.10:40000 → 203.0.113.20:50000: request accepted.
- No direct check completes. A relay allocation provides a reachable pair.
Deterministic IPv4 UDP model: one effective NAT per side, distinct public IPs, filtering from each internal endpoint’s entire outbound history, no hairpinning or port prediction. Four rounds include peer-reflexive triggered checks; TURN is assumed reachable and authorized.
Ben’s carrier NAT may allocate a different endpoint toward Ana than it allocated toward the STUN server. If Ana’s router accepts only the exact endpoint she was told to contact, Ben’s actual packets can be filtered. Two endpoint-dependent mappings with strict filters are especially difficult. But CGNAT always needs TURN
is false: a carrier can use endpoint-independent mappings, and another side’s looser filter can make a direct path possible.
When direct checks fail, TURN, RFC 8656, gives Ben a reachable relay endpoint. Allocation, permissions and optional channel bindings authorize forwarding; a ChannelData message adds a four-byte channel header on the client-to-relay leg. The relay can bridge restrictive access networks, including deployments using TCP or TLS to reach TURN. It forwards the peers’ protected traffic without becoming the DTLS endpoint. Reachability still depends on a working relay service and allowed network path.
Filtering permission belongs to the internal endpoint’s outbound history, separately from the choice of public mapping. If Ben has sent to Ana’s public IP, an address-dependent filter can accept Ana’s packets through any existing binding for that internal endpoint. It does not forget that permission when another destination allocated a new public port. An incoming authenticated check can expose that new port as a peer-reflexive candidate; a triggered check then tests the newly learned route. Two address-dependent mappings with address-dependent filters can therefore succeed even though neither STUN-discovered endpoint completes the first round trip.
A 20-byte question about your address
The check that opens Ana’s route is a binary protocol message, not an HTTP request. STUN, RFC 8489, begins with a 20-byte header: a 16-bit type, a 16-bit body length, a fixed 32-bit magic cookie and a 96-bit transaction ID. Multi-byte numbers use network byte order. The top two type bits are zero, helping distinguish STUN from other traffic sharing the socket.
A Binding Request has type 0x0001; a successful response has type 0x0101. The type encodes method and class in interleaved bit fields, so response types are not formed by simply appending a status byte. The cookie is 0x2112A442. A response echoes the request’s transaction ID, allowing the sender to match it even if several checks are in flight.
After the header come attributes: a 16-bit type, a 16-bit value length, the value, and padding to a four-byte boundary. The attribute’s length excludes padding; the message length includes padded attributes but excludes the initial 20 bytes. Padding is not necessarily zero. That detail matters when checking authentication against existing packets: reconstructing the same semantic attributes with different padding changes the authenticated bytes.
An address disguised with XOR
The IPv4 response in RFC 5769 §2.2 reports 192.0.2.1:32853. In XOR-MAPPED-ADDRESS, a reserved byte is followed by address family 1, the encoded port and four encoded address bytes. XOR the port with the high 16 cookie bits; XOR the IPv4 address with the whole cookie. Applying XOR a second time recovers the original.
0xe112a643 ⊕ 0x2112a442 = 0xc0000201 = 192.0.2.1
This is not encryption. The cookie is public. The encoding avoids some middleboxes rewriting embedded address bytes as if they were ordinary application payload addresses. IPv6 uses 128 bits of mask: the cookie followed by the transaction ID. The transaction identifier’s role therefore extends beyond matching responses.
A public STUN address-discovery exchange and an ICE connectivity check use the same base protocol but different authentication context. ICE peers have shared short-term credentials from signaling. A check’s USERNAME combines the recipient’s fragment and the sender’s fragment; the recipient’s ICE password authenticates the request. The sample request below also carries a priority and an ICE role tie-breaker.
Recompute the RFC’s authentication bytes
| Offset | Bytes | Meaning |
|---|---|---|
| 0–3 | 00 01 00 58 | Type; length excludes 20-byte header |
| 4–7 | 21 12 a4 42 | Magic cookie 0x2112A442 |
| 8–19 | b7 e7 a7 01 bc 34 d6 86 fa 87 df ae | 96-bit transaction ID |
| 20–39 | 80 22 00 10 53 54 55 4e 20 74 65 73 74 20 63 6c 69 65 6e 74 | SOFTWARE |
| 40–47 | 00 24 00 04 6e 00 01 ff | PRIORITY |
| 48–59 | 80 29 00 08 93 2f f9 b1 51 26 3b 36 | ICE-CONTROLLED |
| 60–75 | 00 06 00 09 65 76 74 6a 3a 68 36 76 59 20 20 20 | USERNAME |
| 76–99 | 00 08 00 14 9a ea a7 0c bf d8 cb 56 78 1e f2 b5 b2 d3 f2 49 c1 b5 71 a2 | MESSAGE-INTEGRITY |
| 100–107 | 80 28 00 04 e5 7a 3b cf | FINGERPRINT |
WebCrypto HMAC-SHA1/SHA256 and a local reflected CRC-32 implementation. SHA-256 is selected when present; the other digest comparison remains visible. Unknown required attributes before integrity can reject a cryptographically valid message. Editing a byte checks against the packet’s stored values; restoring returns the exact RFC vector, including padding.
Two checksums with different jobs
MESSAGE-INTEGRITY is HMAC-SHA1 with a 20-byte result. Its input stops before its own attribute header, but the STUN header’s length is temporarily rewritten as though the message ended after the integrity attribute. This avoids circular dependence while authenticating the intended message extent. The lab signs those actual bytes with WebCrypto, then compares the result with the stored digest.
FINGERPRINT is CRC-32, XORed with 0x5354554e, over everything preceding the fingerprint attribute. Here the header carries the full message length, including the eight-byte fingerprint attribute. CRC detects a different class of mistakes and helps identify STUN traffic; anyone can recompute it. It cannot replace HMAC authentication.
A checksum can cover bytes that the authenticated message must ignore. RFC 8489 §9 requires ignoring ordinary attributes after MESSAGE-INTEGRITY, allowing only MESSAGE-INTEGRITY-SHA256 and FINGERPRINT there; after SHA-256 integrity, only FINGERPRINT remains meaningful. Try the tampered preset: an attacker appends a second mapped address after HMAC and recalculates the public CRC. Both comparisons match, but the appended address is outside the authenticated semantics. The receiver keeps the original address and ignores the suffix.
With short-term credentials, SHA-256 controls authentication when both integrity attributes are present; the lab still shows the SHA-1 comparison as a diagnostic. Cryptographic verification and acceptance are separate decisions. An unknown comprehension-required attribute (type 0x0000–0x7fff) before integrity makes a success response unusable even if its selected HMAC and fingerprint match: discard the response and fail the transaction, as required by RFC 8489 §6.3.3. Unknown comprehension-optional attributes may be ignored.
For the sample request the stored fingerprint is e5 7a 3b cf; for the IPv4 response it is c0 7d 4c 96. The response’s software name has a space padding byte after an eleven-byte value. Preserve it to reproduce the published digest. These vectors test the historical SHA-1 integrity attribute that ICE uses; STUN also specifies SHA-256 integrity mechanisms.
Address discovery now has a concrete meaning: Ana sends a transaction, receives her translated endpoint in a checked response, and supplies a candidate to ICE. Connectivity checking adds peer credentials and role/priority attributes to prove that a willing endpoint receives traffic on that path. None of this has yet authenticated Ben’s human identity or encrypted a camera frame.
A reachable peer is not yet a trusted peer
A successful ICE check tells Ana that the holder of the advertised ICE credentials can receive packets. The next question is who holds the cryptographic keys on that route. WebRTC uses DTLS to establish a protected session over the selected datagram path. Unlike an ordinary HTTPS connection, a browser peer often presents a self-signed certificate generated for WebRTC. Trust comes from matching the certificate to the fingerprint conveyed through signaling.
Ana hashes the DER-encoded certificate presented by Ben with the algorithm named in SDP, SHA-256 in this example, and compares the digest with Ben’s advertised fingerprint. The handshake also proves possession of the certificate’s private key. A certificate copied from Ben’s SDP is not sufficient to impersonate him without that key. A certificate with a different key fails the fingerprint comparison.
The DTLS client role is selected through offer/answer setup attributes, independently of ICE’s controlling role. In DTLS 1.2, specified in RFC 6347 (obsoleted by the DTLS 1.3 specification), a handshake uses ClientHello and server handshake messages, certificate/key exchange, and Finished verification. A cookie exchange may precede expensive server state. Handshake records tolerate datagram loss through retransmission and handshake sequencing; they do not inherit TCP’s reliable stream.
DTLS 1.3, RFC 9147, changes handshake messages, encryption and loss recovery. The negotiated version depends on endpoint support; a packet trace must be read according to that version. Describing all WebRTC sessions as a fixed DTLS 1.2 flight sequence would hide those differences. For both versions, authenticated key establishment precedes trusting the resulting media protection context.
Choose a parser before a handshake
Before checking a certificate, the UDP receiver must decide which protocol owns a datagram. RFC 7983, updated by RFC 9443, uses the first byte: 0–3 selects STUN, 16–19 ZRTP, 20–63 DTLS, and 128–191 RTP/RTCP. Values 4–15 are dropped. ZRTP appears in this demultiplexing scheme even though this call establishes media keys with DTLS-SRTP.
The update makes source context matter. Bytes 64–79 select TURN ChannelData when the source IP and port match a responding TURN server; otherwise they select QUIC. Values 80–127 and 192–255 select QUIC. The scheme requires QUIC endpoints not to negotiate fixed-bit greasing. A first byte identifies the next parser, not a valid authenticated packet: a STUN-looking prefix still needs the cookie, lengths and authentication checks, and an RTP-looking prefix still needs the rest of its header and SRTP verification.
One UDP port, several protocol parsers
First byte 128 = 0x80 = 10000000. Forward to RTP/RTCP. This dispatch does not authenticate the packet or prove it is well formed.
RFC 7983 as updated by RFC 9443. Classification chooses a parser; that parser must still validate the complete packet. QUIC multiplexing requires the RFC 9443 fixed-bit constraints.
Who can replace the fingerprint?
Certificate matches SDP. DTLS proves possession of the certificate key; the independent identity check also matches when enabled. Export SRTP keys and accept the peer.
Media uses the keys, not DTLS records
The use_srtp extension negotiates an SRTP protection profile. DTLS-SRTP, RFC 5764, exports keying material with label EXTRACTOR-dtls_srtp and divides it into client/server master keys and salts. Audio and video then travel as SRTP packets directly over UDP, alongside DTLS traffic. They are not wrapped inside one DTLS record per RTP packet.
SRTP, RFC 3711, adds confidentiality, authentication and replay protection to RTP. The AES counter-mode profile combines AES-CM encryption with an HMAC-SHA1 authentication tag; the commonly used SRTP_AES128_CM_SHA1_80 profile has a ten-byte tag. A rollover counter extends the 16-bit RTP sequence into a packet index, helping generate unique encryption inputs across sequence wrap.
AEAD-GCM profiles, RFC 7714, combine authenticated encryption and associated data, with a sixteen-byte tag. The RTP fixed header remains readable and authenticated; payload bytes are encrypted. Some header extensions can receive additional protection under separately negotiated mechanisms. A visible SSRC or sequence number does not imply that the voice samples are visible.
Authentication must succeed before the receiver accepts a packet into its media processing. Replay protection rejects an already accepted packet index; the receiver still permits bounded out-of-order delivery. Packet loss is normal on this path, so replay defense cannot require sequence numbers to arrive strictly consecutively. RTCP has its own protected form, SRTCP, with its own indexing details.
The trust boundary goes through signaling
A packet attacker cannot simply substitute a certificate while SDP remains authentic. A malicious signaling service has a stronger position: it can replace fingerprints and ICE information and arrange two attacker-terminated sessions. Each browser then sees encryption to the key it was told to expect. HTTPS authenticates your connection to that service; it does not prove that the service faithfully named Ben’s key.
Applications that need protection from their signaling operator must bind keys to identity independently, for example through an authenticated out-of-band comparison or an end-to-end identity design. A TURN relay is different: forwarding encrypted packets does not require replacing fingerprints or terminating SRTP. An SFU generally does terminate the browser transport, so protecting content from an SFU requires an additional media encryption layer and key distribution design.
The path, the cryptographic peer and the human identity are three separate things to establish.
Packets with a deadline
Ana’s microphone samples sound. Opus compresses a short interval of those samples, and the sender places the encoded frame in an RTP packet. RTP, RFC 3550, contributes a timeline and source identity, not guaranteed delivery. Sequence numbers tell Ben which packets are missing or reordered. Timestamps tell him when the media belongs on the playout timeline.
The fixed header is twelve bytes. Byte zero contains version V (two bits), padding P (one), extension X (one), and CSRC count CC (four). Byte one contains marker M and the seven-bit payload type PT. Next come the 16-bit sequence number, 32-bit timestamp and 32-bit synchronization source identifier, SSRC. All multi-byte integers use big-endian order.
CC counts optional four-byte contributing source identifiers after the fixed header, which are relevant to mixed streams. If X is set, an extension header follows the CSRC list: a profile identifier and a length in 32-bit words, then that many words of extension data. Negotiated extension formats carry items such as MID, audio level or a transport-wide sequence number. P means padding ends the packet, with its count in the final byte.
With PT 111, a packet beginning 80 6f has V=2, no padding, no extension, no CSRCs and marker 0. Setting marker produces 80 ef. The marker’s meaning depends on the payload format. For Opus it marks a talkspurt boundary; it is not a universal last fragment
flag. For common video payload formats, marker identifies the last packet of a frame.
Build a packet, then distrust its header
V=2 P=0 X=0 CC=0 M=0 PT=111 Sequence 65535 · timestamp 48000 · SSRC 0x12345678 Header 12 bytes · payload 3 bytes · padding 0 bytes
Fixed payload f8 ff fe is an Opus silence packet. Changing PT relabels these bytes; it does not transcode them. The decoder also checks CSRC, extension and padding lengths.
Three clocks, not one counter
A sequence number increments per RTP packet and wraps after 65,535. A timestamp advances with media time, not bytes or packets. Opus RTP, RFC 7587, uses a 48 kHz clock regardless of the encoded audio bandwidth. A 20 ms packet advances it by 48,000 × 0.020 = 960. Initial sequence numbers and timestamps are randomized; they need not start at zero.
Ana’s 30-frame/s video uses a 90 kHz RTP clock, so evenly spaced frames differ by 3,000 ticks. A large frame may need several packets, all with the same RTP timestamp and different sequence numbers. A payload descriptor carries codec-specific fragmentation information. VP8’s start-of-partition bit and H.264’s fragmentation-unit start/end bits are different encodings; RTP itself does not know where an H.264 NAL unit starts.
RTCP sender reports relate an RTP timestamp to an NTP-format time, allowing audio and video clocks to be aligned. Receiver reports summarize packet loss, highest received sequence and interarrival jitter. Feedback can ask for repairs sooner than a periodic summary: Generic NACK names missing packets, PLI reports picture loss, and FIR requests a full intra picture under its own rules. Their formats are specified in RFC 4585 and RFC 5104.
Tell Ana exactly what Ben missed
NACK bit 0 names PID + 1; bit 15 names PID + 16
Generic NACK: request retransmission of sequence(s) 4096, 4097, 4099. PID is always missing; each set BLP bit adds PID + bit + 1 modulo 65,536.
| Offset | Bytes | Meaning |
|---|---|---|
| 0 | 81 | V=2, P=0, FMT=1 |
| 1 | cd | PT=205: RTP feedback |
| 2–3 | 00 03 | 3 → 16 bytes |
| 4–7 | 90 ab cd ef | Feedback sender SSRC |
| 8–11 | 87 65 43 21 | Media source SSRC (zero for FIR) |
| 12–13 | 10 00 | PID: first missing sequence 4096 |
| 14–15 | 00 05 | BLP: 0000000000000101 (bit 0 = PID + 1) |
One Generic NACK entry or one FIR target when building; the decoder accepts multiple entries. Sequence arithmetic wraps at 65,536. The builder starts the FIR request sequence at 1; decoding reads each entry’s value, distinct from an RTP sequence.
For Ana’s SSRC 0x87654321, Ben’s Generic NACK starts 81 cd 00 03: version 2, padding 0, feedback format 1, packet type 205 and length 3. RTCP measures length in 32-bit words minus one, so this packet is sixteen bytes. The next eight bytes name Ben’s feedback sender SSRC 0x90abcdef and Ana’s media source. Its four-byte feedback entry is 10 00 00 05: PID 4096 and bitmask 5, binary 0000000000000101. Bit zero names PID + 1, bit two names PID + 3, so Ben requests packets 4096, 4097 and 4099. A zero bit makes no loss assertion; it is not a positive acknowledgment.
A PLI has packet type 206, format 1 and no feedback control information, so this example is twelve bytes. It reports that picture information was lost; it does not request named packets or guarantee a particular keyframe response. FIR uses type 206, format 4 and an eight-byte entry containing the target SSRC, an eight-bit request sequence and three reserved bytes. The common media-source SSRC is zero for FIR. FIR explicitly requests a full intra picture, useful when switching to a source without the required decoder state. Loss recovery normally uses PLI rather than treating FIR as a stronger periodic NACK. Repeating a FIR request keeps its sequence; a new request advances it, allowing the sender to recognize duplicate requests.
A packet can be correct and still be too late
Packets sent twenty milliseconds apart may arrive one millisecond apart after sitting in a Wi-Fi queue. Ben’s jitter buffer trades a little delay for steadier playout. It reorders packets and waits until each frame’s deadline. If a frame is still missing, the decoder conceals the loss or uses available redundancy. Playing it after that deadline would move the whole conversation backward.
Give a late packet somewhere to land
Buffer 60 ms · jitter 0–40 ms · RTT 60 ms 10 played on first arrival; 0 recovered; 2 concealed. Nominal one-way playout: 90 ms plus capture, encoding and device delay.
12 × 20 ms audio packets, fixed 30 ms base transit, packets 3 and 8 dropped. The first higher-sequence arrival detects a gap; a retransmission costs one RTT. No FEC or adaptive buffer in this model.
NACK is useful only if feedback and repair can beat the deadline. The receiver usually notices loss when a later packet reveals a gap, so detection time matters too. A missing packet near the end of a quiet period may be discovered late. An RTP retransmission stream can use a separate payload type and SSRC; the original sequence is carried in its payload so the receiver can restore the missing media packet.
Forward error correction spends bandwidth before it knows whether a loss will occur. Opus can carry information that helps recover an earlier frame; decoding that information also imposes timing requirements. Packet-loss concealment synthesizes plausible output without getting the original packet back. A browser can combine those tools with an adaptive jitter buffer rather than blindly retransmitting every hole.
The jitter estimate in an RTCP report is not simply the buffer size. It is a smoothed statistic of changes in relative transit time, measured in the RTP clock’s units. The actual playout controller also considers reorder depth, decode time, clock drift and recent underruns. Ana hears the result as continuity, even while a packet capture shows gaps.
Real-time delivery means arriving before playout, not eventually delivering every byte.
Keep the queue short
Ben steps into a crowded train. Available uplink capacity falls, but his camera is still producing the same bitrate. For a while the packets do not disappear; they accumulate in a queue. Video becomes old before it becomes visibly broken. Increasing the jitter buffer treats the symptom by making the call still later. The sender needs to reduce offered load.
Congestion control closes a feedback loop around the network. The sender chooses a target bitrate, the encoder adjusts quality or frame rate, a pacer spreads packets rather than releasing each frame as one burst, and the receiver reports arrivals and losses. RFC 8836 describes the requirements for this real-time setting. WebRTC does not mandate a single immutable congestion algorithm with one universal set of thresholds.
GCC, Google Congestion Control, is an influential implementation family. Its delay-based estimator looks for growing interarrival delay: compare spacing at the sender and receiver, smooth the variation, then infer whether queues are building. A loss-based estimator supplies additional evidence or limits. Modern implementations have evolved beyond the original draft’s precise loss thresholds, so copying a three-line historical formula would not describe every browser’s current behavior.
A useful mental experiment is a 100 kbit/s overshoot. Every second, roughly 100 kbit of extra data joins the queue. On a 1 Mbit/s bottleneck that adds about 100 ms of queueing delay per second until something drops or the sender backs off. Packet pacing reduces bursts; it cannot make a sustained bitrate larger than the bottleneck fit through it.
Transport-wide congestion feedback assigns sequence numbers across the transport and reports received packets and arrival deltas. That lets a sender reason about the shared path across audio, video, padding and repair packets rather than treating every SSRC as an independent pipe. The SDP example offers the widely deployed transport-cc extension URI; that URI names a draft mechanism, not a claim that all browsers negotiate one final RFC format. RFC 8888 defines a standardized RTCP congestion-feedback format with a related purpose.
gi = 0.7 gi−1 + 0.3 δi
Watch feedback drain a bottleneck queue
Sender and bottleneck · shared 0–3,000 kbit/s scale
- Sender
- Bottleneck
Queue waiting time · 0–1940 ms
Time 7.0 s · sender 710 kbit/s · capacity 700 kbit/s Queue 1935.1 ms · smoothed delay gradient 75.93 ms/sample Last decision: await feedback.
| Feedback sample | Value |
|---|---|
| Last acknowledged transport sequence | 810 |
| Sender timestamp | 5399.04 ms |
| Receiver arrival timestamp | 6856.30 ms |
| Send delta Δs | 99.62 ms |
| Arrival delta Δa | 164.57 ms |
| Raw delay change: Δa − Δs | 64.95 ms |
| Filtered delay change | 75.93 ms |
Simplified GCC-style detector, not browser GCC: 1,200-byte packets, unbounded FIFO serializer, 30 ms each-way base delay, feedback per 100 ms send group, no loss. Smooth delay deltas with 0.7/0.3 weights; above 2 ms for three samples cuts rate 15%; below −2 ms holds while draining; otherwise add 20 kbit/s. Capacity is 1,500 kbit/s outside 4–10 s. No autoplay.
The congestion lab sends 1,200-byte packets through a FIFO link. Between four and ten seconds the capacity changes to your chosen value, then returns to 1,500 kbit/s. The link accounts for capacity changes even while a packet is being serialized. Receiver feedback names the transport-wide sequence at the end of each send group and its arrival time; feedback itself takes time to return. Sender spacing and receiver spacing reveal a delay change without synchronizing the two clocks, because a constant clock offset cancels in the differences.
A growing queue stretches arrival spacing relative to send spacing, making the delay gradient positive. Three overuse samples trigger a multiplicative rate reduction in this simplified detector. A negative gradient means packets are catching up as the queue drains; a stable gradient allows additive probing. The graph exposes the delay in this loop: the sender can continue overshooting while its feedback is still in flight. This is a teaching controller with explicit thresholds, not a reproduction of browser GCC’s trendline estimator, probing, loss estimator and experiment-dependent tuning.
A two-person topology stops scaling
Ana invites four colleagues. A mesh with N people has N(N − 1)/2 peer connections. If everybody sends one full-rate stream to everybody else, each uplink carries (N − 1)R and each downlink receives the same. The service saves media-server bandwidth, but Ben’s phone encodes or sends multiple copies, handles many remote decoders, and competes with its own outbound streams.
When Ben invites the whole team
| Topology | Each uplink | Each downlink | Server egress |
|---|---|---|---|
| Mesh | 7.5 Mbit/s | 7.5 Mbit/s | 0 |
| SFU | 1.5 Mbit/s | 7.5 Mbit/s | 45.0 Mbit/s |
| MCU | 1.5 Mbit/s | 1.5 Mbit/s | 9.0 Mbit/s |
6 participants, 1.5 Mbit/s each. Mesh has 15 peer connections; the SFU forwards 30 stream copies.
Equal encoded rates, all streams subscribed, no headers or retransmissions. MCU sends one composite at the chosen rate; SFU forwards one encoding per sender. Real simulcast adds sender encodings.
An Selective forwarding unit: a media server that receives encoded streams and chooses which packets or layers to forward to each participant. changes the topology. Each browser sends to a server, which forwards selected encoded streams. In the simple one-encoding, all-subscribed model, each participant uploads R but still downloads (N − 1)R; the server sends N(N − 1)R in aggregate. An SFU reduces client uplink work, not the mathematical cost of every receiver watching every stream at full quality.
Simulcast gives the SFU several independently encoded versions of one camera, such as thumbnail, medium and large. Ben can receive low-rate thumbnails and one larger active-speaker stream. Sending multiple encodings increases Ana’s upload and encoding work. Switching a receiver’s subscription can then change network use without asking the source to reconstruct a brand-new encoder for every viewer.
Scalable video coding: a coded video stream with layers, where higher layers depend on lower layers and can be omitted for lower quality. organizes temporal or spatial quality into dependent layers within a scalable encoding structure. Dropping a temporal enhancement layer reduces frame rate; dropping a spatial enhancement layer reduces resolution when the codec and negotiated mode permit it. An SFU must respect layer dependencies and switching points. The WebRTC SVC API specification describes how scalability modes are exposed; availability depends on codec and implementation.
An MCU decodes, combines and re-encodes media, giving Ben one composite stream. That saves decoder and downlink work at the client while spending server compute, adding processing delay and choosing a layout on the server. The single composite bitrate in the calculator is an assumption you can set, not a guarantee that six camera tiles preserve their original quality at the bitrate of one camera.
Transport encryption usually ends at the SFU or MCU. Ana’s first two-person call can use a browser-to-browser security association; a group call to a server has a different trust boundary even when the API still says RTCPeerConnection. Topology is an application decision, independent of whether ICE needed TURN on a particular access network.
Chat takes a different branch
Ana types Can you hear me?
while the video bitrate changes. Chat should survive a lost packet, and nobody wants half a UTF-8 sentence delivered as if it were a complete message. The data channel therefore takes a different transport branch: SCTP messages inside DTLS, inside the selected ICE path. Media’s RTP/SRTP packets remain alongside it on the same underlying transport.
RFC 8831 defines the data-channel architecture. SCTP provides message boundaries, multiple streams, reliability and congestion control. DTLS protects the whole SCTP packet. The network does not need to pass native SCTP through Ana’s router; it sees UDP on the direct path. That encapsulation is one reason data channels can share WebRTC’s NAT traversal machinery.
createDataChannel("chat") normally starts in-band negotiation using DCEP, RFC 8832. Ana sends a DATA_CHANNEL_OPEN message, type 0x03, containing the channel type, priority, reliability parameter, label length and protocol length, then the UTF-8 strings. Ben acknowledges with a one-byte DATA_CHANNEL_ACK, 0x02. The DCEP messages use SCTP payload protocol identifier 50.
A channel pairs a stream in each direction with the same stream ID. The DTLS client allocates even IDs and the DTLS server odd IDs when opening in-band channels, avoiding collisions. The label is a name, not a uniqueness constraint: two peers can independently create two channels both called chat
. Ana’s application should designate one creator and have Ben attach to the incoming datachannel event.
Encode a real DATA_CHANNEL_OPEN
03 00 01 00 00 00 00 00 00 04 00 00 63 68 61 74
Type 0x00 · ordered · reliable Label 4 UTF-8 bytes; protocol 0 bytes. 16 DCEP bytes before SCTP/DTLS framing. OPEN uses SCTP PPID 50; the acknowledgment is one byte: 02.
| Offset | Field |
|---|---|
| 0 | 03 · DATA_CHANNEL_OPEN |
| 1 | Channel type: reliability + unordered flag |
| 2–3 | Priority (256: normal) |
| 4–7 | Reliability parameter, unsigned 32-bit |
| 8–11 | Label length, protocol length, each 16-bit |
| 12… | UTF-8 label; empty protocol |
After ACK: send an application message
SCTP PPID field: 00 00 00 33 (51) User data: 48 69
PPID 51: 2 UTF-8 byte(s), delivered as one application message.
PPID is a 32-bit field in the SCTP DATA/I-DATA chunk; it is not prepended to the application payload. These bytes are inside DTLS. Empty string uses PPID 56; empty binary uses 57. The receiver discards the dummy payload byte.
After channel establishment, application messages use different SCTP payload protocol identifiers. In RFC 8831 §6.6, PPID 51 denotes a UTF-8 string and 53 binary data. The string Hi
has payload 48 69; its PPID field is 00 00 00 33, decimal 51. That field belongs to SCTP’s DATA or I-DATA framing, outside the user message bytes.
SCTP cannot send a zero-length user message directly. WebRTC sends a single dummy byte 00 with PPID 56 for an empty string or PPID 57 for empty binary data. The receiver ignores that byte and delivers an empty value of the correct type. A nonempty binary message containing one zero byte also has payload 00, but uses PPID 53; the PPID is what distinguishes meaningful data from empty-message framing. Deprecated PPIDs 52 and 54 belong to an older fragmentation scheme and are not used for new messages here.
Ordering and reliability are independent
The default channel is reliable and ordered. An unordered reliable channel still repairs loss but permits subsequent messages to be delivered before earlier ones. A partially reliable channel lets the sender abandon a message after a retransmission limit or a lifetime. You may set maxRetransmits or maxPacketLifeTime, but not both. A zero retransmission limit still permits the initial transmission.
For chat, reliable ordered delivery is sensible. For rapidly replaced pointer positions, ordered: false with a short lifetime can be better: the newest position matters more than repairing an old one. Partial reliability does not guarantee arrival within that lifetime; it limits how long the sender tries. Ordered partial-reliability streams can advance past abandoned messages through SCTP’s partial-reliability mechanisms.
The lab’s reliability field is four bytes on the wire. The browser API’s two partial-reliability options are unsigned-short limits, so its control deliberately accepts only 0 through 65,535. The unordered flag contributes 0x80; retransmission-limited and lifetime-limited types contribute 0x01 and 0x02. A reliable ordered chat
OPEN is sixteen bytes with a four-byte label and an empty protocol string.
// Reuse the channel passed in by the join action or datachannel event.
function attachChat(chat) {
chat.bufferedAmountLowThreshold = 65536;
chat.onbufferedamountlow = () => enableChatInput();
chat.onopen = () => { status.textContent = "Chat is open."; };
chat.onerror = () => { status.textContent = "Chat transport failed."; };
chat.onclose = () => { status.textContent = "Chat closed. Rejoin to retry."; };
chat.onmessage = ({ data }) => showChatMessage(data);
chatSendButton.onclick = () => {
const text = chatInput.value;
try {
const size = new TextEncoder().encode(text).byteLength;
const limit = pc.sctp?.maxMessageSize ?? 0;
if (chat.readyState !== "open") throw new Error("Chat is not open");
if (size > 16384 || (limit && size > limit)) {
throw new Error("Split this message into smaller chunks");
}
if (chat.bufferedAmount + size > 262144) {
throw new Error("Wait for chat to drain");
}
chat.send(text);
chatInput.value = "";
} catch (error) {
status.textContent = error.message; // keep unsent text for retry
}
};
}
// For expiring position updates instead of chat:
// pc.createDataChannel("position", { ordered: false, maxPacketLifeTime: 100 });
// Never set maxPacketLifeTime and maxRetransmits together.Message boundaries do not make message size unlimited. SDP advertises the largest SCTP user message each receiver accepts. In RFC 8841, absence of max-message-size implies 65,536 bytes; zero expresses no imposed receive limit. The browser’s pc.sctp.maxMessageSize exposes the resulting send limit. Still split large application objects into bounded chunks, especially when a file transfer shares the association with interactive chat.
bufferedAmount reports queued application bytes, and bufferedamountlow tells you when that queue crosses downward through your threshold. Backpressure belongs in the application: disable or pause producers, keep unsent input available, and resume when capacity returns. Calling send() before open, after closing, or with an oversized message can throw; a click handler should announce the failure rather than discard the user’s text.
Separate SCTP streams reduce application-level head-of-line blocking between channels, but they still share transport resources and association congestion control. Very large messages can monopolize transmission on implementations or paths without effective message interleaving. Media and data also compete at the access link. A data channel is a useful peer messaging tool, not permission to fill every available queue.
One packet, every byte accounted for
Take one packet from Ana’s camera after ICE selected the direct path. Her VP8 encoder has produced part of a video frame. The packetizer adds a VP8 payload descriptor; RTP adds sequence, timestamp and SSRC; SRTP encrypts the payload and appends authentication; UDP and IP route the result. The router rewrites the outer source endpoint, not the negotiated RTP identity.
Our concrete RTP header is 80 e0 10 00 00 01 5f 90 87 65 43 21. Version 2, no padding or extensions, marker 1 and payload type 96; sequence 4096, timestamp 90000, SSRC 0x87654321. Ben interprets PT 96 as VP8 because of the negotiated SDP. A different call could assign a different number to the same codec.
The one-byte VP8 descriptor 0x10 has the start-of-partition bit set and partition ID zero, with no optional extended descriptor. In this example the packet also ends the frame, so the RTP marker is set. After SRTP encryption, that descriptor and the compressed video bytes are ciphertext. A packet capture without keys can still read the fixed RTP header and count lengths, but cannot inspect the VP8 frame content.
Budget one VP8 video packet
IP (20 bytes) UDP: 9c 40 c3 50 04 07 [checksum: 2 bytes] RTP: 80 e0 10 00 00 01 5f 90 87 65 43 21 Encrypted: [VP8 descriptor 10 + 1000 video bytes] Authentication: [10 bytes]
Encoded video 1000 B + overhead 51 B = 1051 IP bytes. Overhead 4.85% of the IP packet. At 50 packets/s: 420.4 kbit/s. UDP ports: 40000 → 50000. PT 96, marker 1, sequence 4096, timestamp 90000, SSRC 0x87654321.
Direct UDP path; no CSRCs, RTP extensions, MKI, IP options or link-layer overhead. VP8 has a 1-byte payload descriptor. Ciphertext and authentication tag are shown symbolically.
At the default settings, 1,000 encoded video bytes become 1,051 IPv4 bytes: 20 IP, 8 UDP, 12 RTP, 1 VP8 descriptor and 10 authentication tag bytes around the video. That is 51 bytes of overhead, about 4.85% of the IP packet. IPv6 adds twenty bytes to this base header budget. Switching from the ten-byte AES-CM tag to a sixteen-byte GCM tag adds six more.
This count stops at IP. Ethernet or Wi-Fi adds its own framing, and access networks may add tunnels. RTP extensions, retransmission headers, redundancy, IP options and IPv6 extension headers change the budget. TURN ChannelData adds framing on the client/relay leg. A TURN-over-TLS connection brings TLS and TCP overhead as well as their delivery behavior; it is not the same twelve-byte RTP header suddenly growing a TLS field.
Packet size is constrained by the path MTU, not by the size of a video frame. Packetization splits a large frame into independently routed datagrams so a packet fits without IP fragmentation. Losing one fragment of an IP-fragmented datagram loses that datagram; losing a codec packet can also make part or all of a video frame undecodable, but codec packetization gives the media engine clearer repair and loss decisions.
At 50 packets per second, a ten-byte-per-packet choice costs 4 kbit/s. For sparse audio that overhead is a larger fraction of traffic than for full video packets. Larger audio packets amortize headers but add capture delay and make one loss cover more sound. The media engine continually balances such tradeoffs; fewer packets
is not automatically better for a conversation.
The path is a lease, not a permanent fact
Ben leaves Wi-Fi for mobile data. His interface and public mapping change, so the selected pair may stop working. pc.restartIce() requests new ICE credentials and triggers negotiation; an offer carrying the restart starts a new ICE generation. Both sides exchange fresh candidates and check routes again. The existing path can continue while a replacement is established if it still works.
Restarting ICE is different from starting an entirely new application call. Existing tracks and transceivers can remain, and the browser manages the transport transition. Your signaling code must recognize generations so a late candidate from the old network does not contaminate the new checklist. A connection may pass through disconnected temporarily; treating every transient state as an immediate permanent hangup can interrupt a recoverable call.
Even an unchanged address needs ongoing permission to receive. Consent freshness, RFC 7675, sends authenticated STUN Binding requests periodically. The recommended default is five seconds, randomized to four through six seconds between checks. If no valid response arrives for thirty seconds, the sender must stop transmitting application traffic on that five-tuple. A fresh successful check before expiry maintains consent; ordinary media packets alone do not.
This prevents a browser from continuing to flood an endpoint that has stopped consenting. Keeping NAT state alive is useful too, but consent and NAT maintenance are different purposes. A relay allocation, a successful DTLS handshake and an old ICE nomination do not grant an unlimited lifetime of permission.
Follow Ana’s next syllable all the way down. A microphone capture interval becomes an Opus frame, gets an RTP timestamp, is authenticated and encrypted with exported SRTP keys, then crosses a route ICE proved reachable. Ben authenticates the packet, places it against a playout deadline, decodes and sends samples to the speaker. Meanwhile chat takes SCTP inside DTLS over the same path. The signaling server arranged the meeting; these independent mechanisms keep the conversation alive.
A call is a continuing agreement: compatible media, a tested path, authenticated keys, congestion feedback and fresh consent.
End of WebRTC · 10 levels, Idea to Machine
Where to next
Next in Web and networking · 09 of 12
Video streaming, down to the decoder
Press play on a ten-minute talk. Follow the first picture, the next segment and a sudden tunnel through HLS, DASH, adaptive bitrate, browser buffers and the silicon that turns bytes into motion.