Audio/Video Transport Core Maintenance (avtcore) Working Group =============================================================== CHAIRS: Jonathan Lennox Bernard Aboba Virtual Interim Agenda Date: Thursday, February 23, 2023 Time: 08:00 - 10:00 Pacific Time Notes: https://notes.ietf.org/s/notes-ietf-interim-2023-avtcore-01-avtcore Meeting link: https://datatracker.ietf.org/meeting/interim-2023-avtcore-01/session/avtcore Remote instructions: https://meetings.conf.meetecho.com/interim/?short=4da9b4c1-62cd-458e-8a8f-58fa91cf8ce4 Slides: https://docs.google.com/presentation/d/1QAo7WiUmIfKWYp5ntV37aYvtnSuyZ0gNqrcrnQS4KkM/ ------------------------------------------------- 1. Preliminaries (Chairs, 15 min) Note Well, Note Takers, Agenda Bashing, Draft status 2. RTP Payload Format for SCIP (D. Hanson, 15 min) https://datatracker.ietf.org/doc/html/draft-ietf-avtcore-rtp-scip 3. RTP Control Protocol (RTCP) Messages for Green Metadata (Yong He, 15 min) https://datatracker.ietf.org/doc/html/draft-ietf-avtcore-rtcp-green-metadata 4. RTP over QUIC (J. Ott, M. Engelbart, S. Dawkins, 25 min) https://datatracker.ietf.org/doc/html/draft-ietf-avtcore-rtp-over-quic 5. Viewport and Region-of-Interest-Dependent Delivery of Visual Volumetric Media (S. Gudumasu, 10 min) https://datatracker.ietf.org/doc/html/draft-gudumasu-avtcore-rtp-volumetric-media-roi 6. Wrapup and Next Steps (Chairs, 15 min) ------------------------------------------------- # Draft status updates * VP9 payload is in MISSREF, waiting on framemarking draft. # RTP Payload Format for SCIP (Dan Hanson, Mike Faller) IESG Ballot has completed, 3 DISCUSS comments. * In SCIP packetization and rate control is handled by the underlying codec. If the underlying codec (e.g. G.729) does not have the ability to control rate, then the ability of SCIP to respond to congestion will be limited. * SCIP supports both control and data traffic. Control traffic handles key management as well as negotiation. Since control traffic has structure as defined in the SCIP documents and includes a length field, so that SCIP control messages can be split between RTP packets. Data traffic consists of audio/video as well as chat and whiteboard, is encrypted and therefore appears to RTP as an opaque blob that can be split between RTP packets. This means that the SCIP stream can appear differently depending on the state of SCIP and whether the traffic represents control or data. * SCIP authors are looking for WG review of the proposed changes before publishing a revision 5. * Bernard Aboba: Would be useful to take a look at the ballot positions. The IESG is looking for a standard RTP payload type document, with sections on the RTP packet format, RTCP feedback, etc. but they didn't get that. * BA: The SCIP authors appear to have responded to Franchesca's review comments relating to change control. Roman is asking if the traffic is entirely opaque - once you've exchanged the keys, it is entirely opaque, correct? * Dan Hanson: Yes. But the control traffic has structure, as described in SCIP-210. * BA: While you don't have to go into detail on the key establishment process, I do think you need to describe how control traffic and data traffic are packetized. Once keys are negotiated, the data traffic is entirely opaque. Packetization is handled by the underlying codec, which hands the payload up to SCIP for encryption and placement in the payload field. So if the codec is H.264, an H.264 packetizer is providing a payload in RFC 6184 format to SCIP for encryption. * BA: You may need to say a little bit about how the SCIP are fragmented into RTP packets. * DH: The messages have their own header with message length. * BA: So you will need to describe how the SCIP control messages are split between packets (e.g. how fragmentation works) as well as how the fragments are de-packetized. * BA: There is another comment from Roman about secure session establishment protocol behaviour. I'd suggest talking about the basics, but I don't think you need to reproduce the SCIP state machine, because you just need enough to describe how packetization and de-packetization works. * BA: Sarker's comment asks what RTP profile should be used. Also need to specify exactly what "highly variable" means. * Jonathan Lennox: The RTP profile will depend on the underlying codec. * BA: However, SDP does not reflect that, because the negotiation is handled within SCIP. * Mike Faller: We've been throwing the word "codec" around a lot, but that underlying codec could be a chat session, so it's better to think of it as data being carried by SCIP. * BA: Yes, since the payload can represent media or chat or whiteboard, data is probably a better term. * BA: This seems like a question that a description of the overall architecture and layering might address. * MF: What do you need to know about SCIP to implement it? * BA: Since negotiation of the underlying codec is handled by SCIP, the RTP profile does not need to be negotiated in SDP. Also, that is why the traffic is "highly variable". * MF: One of the fears from networking equipment vendors is wanting to know how to filter on SCIP, but you can't as it's so variable and changing. * Jonathan Lennox: Need a statement in the document to just not try. * BA: QUIC deals with this in the "ossification" section - you could add a section here telling implementers to not bother with deep packet inspection because the traffic can vary depending on whether it is data or control traffic. Attempting to parse opaque data is pointless. # RTP Control Protocol (RTCP) Messages for Green Metadata (Yong He) * Jonathan Lennox: Question from Nokia - if there was any negotiation of the resolution in SDP, how does that interact with these messages? Probably that you should never go above the resolution which SDP allows. * JL: Question from Magnus Westerland about the format * JL: Better to call the document "Temporal-Spatial Resolution" as the Green Metadata effort within MPEG is more than just TSR. # RTP over QUIC (Mathis Engelbart, Spencer Dawkins) * Mathis Engelbart: Had feedback that the congestion control section needed to describe what kind of congestion control was required, and which layer should perform congestion control. *Spencer's summary of controversial ideas in congestion control:* 1. *"Disabling QUIC CC"* - obviously, any implementation can do anything at its end, but there's not a defined way to tell the OTHER end to do that. So, how do we make sure we're talking to an "other end" that will Do The Right Thing? Port numbers? ALPN (as is in the draft today)? Just assume the RTP sender will never cause QUIC CC to kick in, because it's interactive media, and think happy thoughts? Other ideas? 2. *Conforming to RFC 8085* (because we're running over UDP with little or no congestion control happening there). 3. *Is there a MTI rate adaptation algorithm for RTP-over-QUIC?* - Spencer doesn't think so, because NADA and SCREAM are IRTF-stream algorithms. For some value of "we", "we" could talk to the authors about how to bring this through the IETF. Are we going to do that? Spencer also notes that Christian Huitema has a proposal for media-aware CC in MOQ now, so it may be too soon to pick an algorithm now.* * Spencer Dawkins: Been talking about disabling QUIC CC for years. Any implementation can do whatever it wants at its end, but no way for the peer to be told to do the right thing. Could be in ALPN, but some people don't like that. * SD: What would we need to do to conform to RFC 8085 as we're running over UDP, and turning off QUIC CC would require understanding of what the app layer will do * Bernard Aboba: In QUIC, both sides don't need to use the same congestion control algorithm. The CC algorithm isn't negotiated, and yet the two sides can interoperate. On disabling QUIC CC, is this a serious thing that people want to do? We've talked in various places about trying to choose CCs that would work. If you are compiling a QUIC library within your application, you could remove the QUIC CC algorithm and handle CC in the application instead, but it's highly unlikely that a browser would allow an application to disable QUIC CC. As an example, in WebTransport, there is no constructor argument for "no congestion control". * SD: There's two layers to this - using the language of disabling QUIC CC, but how do all the entities in an implementation know who is doing congestion control, and how to know that's not appropriate for H3? I.e. don't use BBR or other CCs that do bandwidth probing. * BA: In the case you've just mentioned, it's possible for one peer to use BBR and the other to use New Reno. The Probe RTT phase would destabilize rate control (the application sees loss or increased delay, so it reduces rate, then probe RTT kicks in, delay goes back up and application will reduce rate even more though it is not necessary). So rather than worrying about CC negotiation, the draft should instead describe what algorithms are likely to provide good results. * SD: Should we have a mandatory to implement rate adaptation algorithm? Spencer doesn't think so, but wondering if others agree? * BA: Rate adaptation depends on what mechanisms the encoder provides. The encoder may support scalable video coding, which enables dropping or adding of layers, or it may support per-frame or per-macroblock QP control. Or the encoder might just allow changes in resolution or framerate. So a given rate control algorithm may not be implementable if the encoder doesn't offer the required controls. * SD: A lot of the work in RMCAT (NADA, SCReAM) was done on how to not break other flows of NADA or SCReAM, but not necessarily what would break other congestion controllers. * Peter Thatcher: I don't think this is a question of whether we're turning off QUIC CC - what is the feedback it's going to use to implement feedback to RTP - is that going to be feedback from QUIC or from RTP? Are we going to extend QUIC to have the timestamps necessary, or is it going to be feedback embedded in streams or datagrams. How implementations use that feedback should be up to that implementation. * SD: Agree with Peter. I want to improve the quality of discussions by having better descriptions on issues on github. We haven't had a lot of conversation that focuses on all three options. * Jonathan Lennox: Agree that whether you have congestion control at the RTP level - I think the desision of how much is available to send can happen at either end, but the decision of what to send needs to be decided by the sender only and this needs to be clarified. My expectation is that a CC like New Reno or CUBIC, and if you run a rate-based CC like GCC or NADA on top of that is that they'd work together well, but not with BBR. The issue is communication between the layers. * BA: Key frames cause the most issues because they are so much larger than everything else. Average Bitrate Targets are just averages, but since keyframes are 10+ times larger than P-frames, keyframes are much more susceptible to loss and queuing delays. Also, when sent over QUIC reliable transport, the size of the congestion window is useful to know, since keyframes may require multiple roundtrips to send. For exampe, at high resolutions, the keyframe might be 150 KB and if the cwind is 15 KB (10 packets), that implies 10 roundtrips! This can dictate the startup latency, or the length of a video freeze that will be experienced in event that a keyframe is needed to switch between streams or to recover from loss. W3C WebTransport WG has been working on metrics to provide to the application. * SD: Going to have an arms race between people wanting to send more and those wanting better compression. The very biggest frames that we send are getting bigger over time, until someone comes up with a way to amend that. * BA: Better compression means you can either keep the quality the same and save bits, or use the same number of bits but do more with it. In many applications (such as 4K gaming or AR/VR) the inclination will be to provide higher quality, so the bitrate won't go down. * SD: Mathis and Joerg have already done some due dilligence with QUIC CC, and noted it wasn't kicking in on some representative RTP traffic. It didn't, but we need to keep an eye open in case that starts happening. * BA: The amount of concurrency is an issue. If you're sending the key frame at the same time as P-frames, this allows the pipe to stay full even if the keyframe is experiencing loss and retransmission. While this may drag out the length of time needed to send a key frame, it can decrease glass-glass latency because the pipe is kept full, and the cwind grows or recovers more quickly (since it being pushed up by the concurrent sending of P-frames). # Viewport and Region-of-Interest-Dependent Delivery of Visual Volumetric Media (Srinivas Gudumasu) * Jonathan Lennox: Are you seeking adoption? * Srinivas Gudumasu: Not at this time, just looking for feedback. * Spencer Dawkins: Is this the rtp-v3c-00 draft? * SG: No this is on top of that * JL: Might be better for MPEG to define what a region and the viewport is - make sure you have the correct experts looking at it. * SG: Already working on this. * Mo Zanaty: You were talking about coordinates system - does V3C describe in terms of meters? Or are they arbitrary and not based on real-world sizes? Might be not interoperable when looking at an observer's real world dimensions. * SG: The sender converts the dimensions and translates them into the coordinates for the spatial regions. * Mo Zanaty: Needs more wordsmithing to make this more obvious that it's images and pixels relative to the media itself. # Action items * SCIP authors to add material to the introduction providing an overview of SCIP, section on RTP payload format (including packetization/de-packetization) answering IESG questions. # Log of Zulip Chat Sam Hurst I can try taking notes - will be my first time for an IETF meeting but I can give it a go 11:05:52 Dale Worley I'm not hearing any audio, though the browser and Meetecho think I've got sound enabled and Meetecho reports multiple kbps. I heard the previous speaker. 11:17:11 Jonathan Lennox Dale, is that still the case? 11:19:14 Dale Worley Yes, and toggling the immediately obvious controls doesn't fix it. I am getting the notifications as to who is speaking. 11:20:12 Ugh, my UI problem. The "Mute Audio" icon in the upper left is "mute audio to me" whereas the same icon in the lower right is "mute audio I am generating". 11:22:23 Sam Hurst My audio glitched out for a moment there - what doesn't matter (so I can add it to the notes) 11:33:03 Rui Paulo we can also tell the other side to disable CC via QUIC transport parameters 11:50:17 Joerg Ott That would mean that a QUIC library without RTP would be able to signal that, too, which we may not want to suggest. No CC at all would not be a good idea. 11:51:10 Peter Thatcher This isn't really about "disabling QUIC CC". This is more about "the congestion control for the RTP packets is done using feedback other than QUIC feedback". Whether it's "in QUIC" or "out of QUIC" is an implementation detail. The real question for a standard is: what feedback is being used. Is it QUIC feedback or RTP/RTCP feedback? 11:54:30 Joerg Ott Right, I am saying only that you should not be allowed to negotiate turning off QUIC CC inside QUIC transport parameter negotiation 11:55:39 Peter Thatcher For example, in the WebRTC world, we don't standardize googc. We standardize transport-wide-cc (or whatever was the result of trying to standardize it :). 11:56:01 I agree it doesn't make sense to negotiate turning off QUIC CC. It might make sense to negotiate some kind of QUIC version of REMB, though. 11:56:58 Harald Alvestrand well, we failed to find the resources to actually propose transport-cc for standardization, so that kind of died..... 11:57:10 the proper view is probably that we should have a congestion limitation as part of QUIC, but a decision on what to send next (or decide not to send) as part of the application. 11:58:21 Peter Thatcher Options for feedback: A. REMB style: have the receiver tell the sender a number. Probably need a timestamp similar to the RTP abs-send-time header extension attached to each packet for the receiver to make a good calculation. B. transport-wide-cc style: have the receiver send per-packet receive timestamps in ACKs. Let the sender do the calculation. I think in either case, extending QUIC will be much better than doing this above QUIC. 12:02:25 I think this works for B: https://www.ietf.org/archive/id/draft-smith-quic-receive-ts-00.txt 12:03:52 Joerg Ott B. is what we have in the draft 12:05:15 including quite a bit of discussion on what to inherit from the QUIC layer 12:05:40 Vidhi Goel How big is the Pframe? 12:06:12 Joerg Ott Of course, good ol' RTCP is also an option when you need something that QUIC doesn't give you (yet) 12:06:29 Peter Thatcher Joerg: That sounds like a good approach. Why don't we just try and implement that with, say, googcc, and see if it works well? I'm guessing it will. What more do we need? 12:07:08 Mo Zanaty MOQ has a similar need / delimma. That argues for solving this within the QUIC CC and feedback layer, not above it. 12:07:53 Peter Thatcher And I'd like googc to work in WebTransport as well. 12:08:14 Mo Zanaty For receive timestamp extensions to QUIC, there is also Christian's original proposal: https://datatracker.ietf.org/doc/draft-huitema-quic-ts/ 12:09:10 Peter Thatcher If anyone is interested in adding support for https://www.ietf.org/archive/id/draft-smith-quic-receive-ts-00.txt and googcc to an impl of QUIC in Rust, let me know. I'm interested in making that work. 12:10:30 Joerg Ott Peter: we implemented quite a bit of this, and it seemed to work. But the details of QUIC CC interaction remain. 12:10:43 Let's chat offline about this (I will have to disappear shortly for boarding my flight) 12:11:22 Vidhi Goel I can work on congestion control issues 12:13:29 Peter Thatcher Joerg: I'm interested to see how far you got 12:13:50 If anyone wants to chat QUIC CC offline, send me an email (pthatcher@microsoft.com) 12:15:25 Joerg Ott We'll reach out. 12:15:44