install go2rtc on bob

This commit is contained in:
2026-04-04 19:36:14 +02:00
parent f0b56e63d1
commit ccf88187b8
537 changed files with 69213 additions and 0 deletions
+114
View File
@@ -0,0 +1,114 @@
# Notes
go2rtc tries to name formats, protocols and codecs the same way they are named in FFmpeg.
Some formats and protocols go2rtc supports exclusively. They have no equivalent in FFmpeg.
## Producers (input)
- The initiator of the connection can be go2rtc - **Source protocols**
- The initiator of the connection can be an external program - **Ingress protocols**
- Codecs can be incoming - **Receiver codecs**
- Codecs can be outgoing (two way audio) - **Sender codecs**
| Group | Format | Protocols | Ingress | Receiver codecs | Sender codecs | Example |
|------------|--------------|-----------------|---------|---------------------------------|---------------------|---------------|
| Devices | alsa | pipe | | | pcm | `alsa:` |
| Devices | v4l2 | pipe | | | | `v4l2:` |
| Files | adts | http, tcp, pipe | http | aac | | `http:` |
| Files | flv | http, tcp, pipe | http | h264, aac | | `http:` |
| Files | h264 | http, tcp, pipe | http | h264 | | `http:` |
| Files | hevc | http, tcp, pipe | http | hevc | | `http:` |
| Files | hls | http | | h264, h265, aac, opus | | `http:` |
| Files | mjpeg | http, tcp, pipe | http | mjpeg | | `http:` |
| Files | mpegts | http, tcp, pipe | http | h264, hevc, aac, opus | | `http:` |
| Files | wav | http, tcp, pipe | http | pcm_alaw, pcm_mulaw | | `http:` |
| Net (pub) | mpjpeg | http, tcp, pipe | http | mjpeg | | `http:` |
| Net (pub) | onvif | rtsp | | | | `onvif:` |
| Net (pub) | rtmp | rtmp | rtmp | h264, aac | | `rtmp:` |
| Net (pub) | rtsp | rtsp, ws | rtsp | h264, hevc, aac, pcm*, opus | pcm*, opus | `rtsp:` |
| Net (pub) | webrtc* | webrtc | webrtc | h264, pcm_alaw, pcm_mulaw, opus | pcm_alaw, pcm_mulaw | `webrtc:` |
| Net (pub) | yuv4mpegpipe | http, tcp, pipe | http | rawvideo | | `http:` |
| Net (priv) | bubble | http | | h264, hevc, pcm_alaw | | `bubble:` |
| Net (priv) | doorbird | http | | | | `doorbird:` |
| Net (priv) | dvrip | tcp | | h264, hevc, pcm_alaw, pcm_mulaw | pcm_alaw | `dvrip:` |
| Net (priv) | eseecloud | http | | | | `eseecloud:` |
| Net (priv) | gopro | udp | | TODO | | `gopro:` |
| Net (priv) | hass | webrtc | | TODO | | `hass:` |
| Net (priv) | homekit | hap | | h264, eld* | | `homekit:` |
| Net (priv) | isapi | http | | | pcm_alaw, pcm_mulaw | `isapi:` |
| Net (priv) | kasa | http | | h264, pcm_mulaw | | `kasa:` |
| Net (priv) | nest | rtsp, webrtc | | TODO | | `nest:` |
| Net (priv) | ring | webrtc | | | | `ring:` |
| Net (priv) | roborock | webrtc | | h264, opus | opus | `roborock:` |
| Net (priv) | tapo | http | | h264, pcma | pcm_alaw | `tapo:` |
| Net (priv) | tuya | webrtc | | | | `tuya:` |
| Net (priv) | vigi | http | | | | `vigi:` |
| Net (priv) | webtorrent | webrtc | TODO | TODO | TODO | `webtorrent:` |
| Net (priv) | xiaomi* | cs2, tutk | | | | `xiaomi:` |
| Services | flussonic | ws | | | | `flussonic:` |
| Services | ivideon | ws | | h264 | | `ivideon:` |
| Services | yandex | webrtc | | | | `yandex:` |
| Other | echo | * | | | | `echo:` |
| Other | exec | pipe, rtsp | | | | `exec:` |
| Other | expr | * | | | | `expr:` |
| Other | ffmpeg | pipe, rtsp | | | | `ffmpeg:` |
| Other | stdin | pipe | | | pcm_alaw, pcm_mulaw | `stdin:` |
- **eld** - rare variant of aac codec
- **pcm** - pcm_alaw pcm_mulaw pcm_s16be pcm_s16le
- **webrtc** - webrtc/kinesis, webrtc/openipc, webrtc/milestone, webrtc/wyze, webrtc/whep
## Consumers (output)
| Format | Protocol | Send codecs | Recv codecs | Example |
|--------------|----------|---------------------------------|---------------------------|---------------------------------------|
| adts | http | aac | | `GET /api/stream.adts` |
| ascii | http | mjpeg | | `GET /api/stream.ascii` |
| flv | http | h264, aac | | `GET /api/stream.flv` |
| hls/mpegts | http | h264, hevc, aac | | `GET /api/stream.m3u8` |
| hls/fmp4 | http | h264, hevc, aac, pcm*, opus | | `GET /api/stream.m3u8?mp4` |
| homekit | hap | h264, opus | | Apple HomeKit app |
| mjpeg | ws | mjpeg | | `{"type":"mjpeg"}` -> `/api/ws` |
| mpjpeg | http | mjpeg | | `GET /api/stream.mjpeg` |
| mp4 | http | h264, hevc, aac, pcm*, opus | | `GET /api/stream.mp4` |
| mse/fmp4 | ws | h264, hevc, aac, pcm*, opus | | `{"type":"mse"}` -> `/api/ws` |
| mpegts | http | h264, hevc, aac | | `GET /api/stream.ts` |
| rtmp | rtmp | h264, aac | | `rtmp://localhost:1935/{stream_name}` |
| rtsp | rtsp | h264, hevc, aac, pcm*, opus | | `rtsp://localhost:8554/{stream_name}` |
| webrtc | webrtc | h264, pcm_alaw, pcm_mulaw, opus | pcm_alaw, pcm_mulaw, opus | `{"type":"webrtc"}` -> `/api/ws` |
| yuv4mpegpipe | http | rawvideo | | `GET /api/stream.y4m` |
- **pcm** - pcm_alaw pcm_mulaw pcm_s16be pcm_s16le
## Snapshots
| Format | Protocol | Send codecs | Example |
|--------|----------|-------------|-----------------------|
| jpeg | http | mjpeg | `GET /api/frame.jpeg` |
| mp4 | http | h264,hevc | `GET /api/frame.mp4` |
## Developers
**File naming:**
- `pkg/{format}/producer.go` - producer for this format (also if support backchannel)
- `pkg/{format}/consumer.go` - consumer for this format
- `pkg/{format}/backchannel.go` - producer with only backchannel func
**Mentioning modules:**
- [`main.go`](../main.go)
- [`README.md`](../README.md)
- [`internal/README.md`](../internal/README.md)
- [`website/.vitepress/config.js`](../website/.vitepress/config.js)
- [`website/api/openapi.yaml`](../website/api/openapi.yaml)
- [`www/schema.json`](../www/schema.json)
## Useful links
- https://www.wowza.com/blog/streaming-protocols
- https://vimeo.com/blog/post/rtmp-stream/
- https://sanjeev-pandey.medium.com/understanding-the-mpeg-4-moov-atom-pseudo-streaming-in-mp4-93935e1b9e9a
- [Android Supported media formats](https://developer.android.com/guide/topics/media/media-formats)
- [THEOplayer](https://www.theoplayer.com/test-your-stream-hls-dash-hesp)
- [How Generate DTS/PTS](https://www.ramugedia.com/how-generate-dts-pts-from-elementary-stream)
+20
View File
@@ -0,0 +1,20 @@
## AAC-LD and AAC-ELD
| Codec | Rate | QuickTime | ffmpeg | VLC |
|---------|-------|-----------|--------|-----|
| AAC-LD | 8000 | yes | no | no |
| AAC-LD | 16000 | yes | no | no |
| AAC-LD | 22050 | yes | yes | no |
| AAC-LD | 24000 | yes | yes | no |
| AAC-LD | 32000 | yes | yes | no |
| AAC-ELD | 8000 | yes | no | no |
| AAC-ELD | 16000 | yes | no | no |
| AAC-ELD | 22050 | yes | yes | yes |
| AAC-ELD | 24000 | yes | yes | yes |
| AAC-ELD | 32000 | yes | yes | yes |
## Useful links
- [4.6.20 Enhanced Low Delay Codec](https://csclub.uwaterloo.ca/~ehashman/ISO14496-3-2009.pdf)
- https://stackoverflow.com/questions/40014508/aac-adts-for-aacobject-eld-packets
- https://code.videolan.org/videolan/vlc/-/blob/master/modules/packetizer/mpeg4audio.c
+126
View File
@@ -0,0 +1,126 @@
package aac
import (
"encoding/hex"
"fmt"
"github.com/AlexxIT/go2rtc/pkg/bits"
"github.com/AlexxIT/go2rtc/pkg/core"
)
const (
TypeAACMain = 1
TypeAACLC = 2 // Low Complexity
TypeAACLD = 23 // Low Delay (48000, 44100, 32000, 24000, 22050)
TypeESCAPE = 31
TypeAACELD = 39 // Enhanced Low Delay
AUTime = 1024
// FMTP streamtype=5 - audio stream
FMTP = "streamtype=5;profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config="
)
var sampleRates = [16]uint32{
96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350,
0, 0, 0, // protection from request sampleRates[15]
}
func ConfigToCodec(conf []byte) *core.Codec {
// https://en.wikipedia.org/wiki/MPEG-4_Part_3#MPEG-4_Audio_Object_Types
rd := bits.NewReader(conf)
codec := &core.Codec{
FmtpLine: FMTP + hex.EncodeToString(conf),
PayloadType: core.PayloadTypeRAW,
}
objType := rd.ReadBits(5)
if objType == TypeESCAPE {
objType = 32 + rd.ReadBits(6)
}
switch objType {
case TypeAACLC, TypeAACLD, TypeAACELD:
codec.Name = core.CodecAAC
default:
codec.Name = fmt.Sprintf("AAC-%X", objType)
}
if sampleRateIdx := rd.ReadBits8(4); sampleRateIdx < 0x0F {
codec.ClockRate = sampleRates[sampleRateIdx]
} else {
codec.ClockRate = rd.ReadBits(24)
}
codec.Channels = rd.ReadBits8(4)
return codec
}
func DecodeConfig(b []byte) (objType, sampleFreqIdx, channels byte, sampleRate uint32) {
rd := bits.NewReader(b)
objType = rd.ReadBits8(5)
if objType == 0b11111 {
objType = 32 + rd.ReadBits8(6)
}
sampleFreqIdx = rd.ReadBits8(4)
if sampleFreqIdx == 0b1111 {
sampleRate = rd.ReadBits(24)
} else {
sampleRate = sampleRates[sampleFreqIdx]
}
channels = rd.ReadBits8(4)
return
}
func EncodeConfig(objType byte, sampleRate uint32, channels byte, shortFrame bool) []byte {
wr := bits.NewWriter(nil)
if objType < TypeESCAPE {
wr.WriteBits8(objType, 5)
} else {
wr.WriteBits8(TypeESCAPE, 5)
wr.WriteBits8(objType-32, 6)
}
i := indexUint32(sampleRates[:], sampleRate)
if i >= 0 {
wr.WriteBits8(byte(i), 4)
} else {
wr.WriteBits8(0xF, 4)
wr.WriteBits(sampleRate, 24)
}
wr.WriteBits8(channels, 4)
switch objType {
case TypeAACLD:
// https://github.com/FFmpeg/FFmpeg/blob/67d392b97941bb51fb7af3a3c9387f5ab895fa46/libavcodec/aacdec_template.c#L841
wr.WriteBool(shortFrame)
wr.WriteBit(0) // dependsOnCoreCoder
wr.WriteBit(0) // extension_flag
wr.WriteBits8(0, 2) // ep_config
case TypeAACELD:
// https://github.com/FFmpeg/FFmpeg/blob/67d392b97941bb51fb7af3a3c9387f5ab895fa46/libavcodec/aacdec_template.c#L922
wr.WriteBool(shortFrame)
wr.WriteBits8(0, 3) // res_flags
wr.WriteBit(0) // ldSbrPresentFlag
wr.WriteBits8(0, 4) // ELDEXT_TERM
wr.WriteBits8(0, 2) // ep_config
}
return wr.Bytes()
}
func indexUint32(s []uint32, v uint32) int {
for i := range s {
if v == s[i] {
return i
}
}
return -1
}
@@ -0,0 +1,52 @@
package aac
import (
"encoding/hex"
"testing"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/stretchr/testify/require"
)
func TestConfigToCodec(t *testing.T) {
s := "profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=F8EC3000"
s = core.Between(s, "config=", ";")
src, err := hex.DecodeString(s)
require.Nil(t, err)
codec := ConfigToCodec(src)
require.Equal(t, core.CodecAAC, codec.Name)
require.Equal(t, uint32(24000), codec.ClockRate)
require.Equal(t, uint16(1), codec.Channels)
dst := EncodeConfig(TypeAACELD, 24000, 1, true)
require.Equal(t, src, dst)
}
func TestADTS(t *testing.T) {
// FFmpeg MPEG-TS AAC (one packet)
s := "fff15080021ffc210049900219002380fff15080021ffc212049900219002380" //...
src, err := hex.DecodeString(s)
require.Nil(t, err)
codec := ADTSToCodec(src)
require.Equal(t, uint32(44100), codec.ClockRate)
require.Equal(t, uint16(2), codec.Channels)
size := ReadADTSSize(src)
require.Equal(t, uint16(16), size)
dst := CodecToADTS(codec)
WriteADTSSize(dst, size)
require.Equal(t, src[:len(dst)], dst)
}
func TestEncodeConfig(t *testing.T) {
conf := EncodeConfig(TypeAACLC, 48000, 1, false)
require.Equal(t, "1188", hex.EncodeToString(conf))
conf = EncodeConfig(TypeAACLC, 16000, 1, false)
require.Equal(t, "1408", hex.EncodeToString(conf))
conf = EncodeConfig(TypeAACLC, 8000, 1, false)
require.Equal(t, "1588", hex.EncodeToString(conf))
}
+148
View File
@@ -0,0 +1,148 @@
package aac
import (
"encoding/hex"
"github.com/AlexxIT/go2rtc/pkg/bits"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/pion/rtp"
)
const ADTSHeaderSize = 7
func ADTSHeaderLen(b []byte) int {
if HasCRC(b) {
return 9 // 7 bytes header + 2 bytes CRC
}
return ADTSHeaderSize
}
func IsADTS(b []byte) bool {
// AAAAAAAA AAAABCCD EEFFFFGH HHIJKLMM MMMMMMMM MMMOOOOO OOOOOOPP (QQQQQQQQ QQQQQQQQ)
// A 12 Syncword, all bits must be set to 1.
// C 2 Layer, always set to 0.
return len(b) >= ADTSHeaderSize && b[0] == 0xFF && b[1]&0b1111_0110 == 0xF0
}
func HasCRC(b []byte) bool {
// AAAAAAAA AAAABCCD EEFFFFGH HHIJKLMM MMMMMMMM MMMOOOOO OOOOOOPP (QQQQQQQQ QQQQQQQQ)
// D 1 Protection absence, set to 1 if there is no CRC and 0 if there is CRC.
return b[1]&0b1 == 0
}
func ADTSToCodec(b []byte) *core.Codec {
// 1. Check ADTS header
if !IsADTS(b) {
return nil
}
// 2. Decode ADTS params
// https://wiki.multimedia.cx/index.php/ADTS
rd := bits.NewReader(b)
_ = rd.ReadBits(12) // Syncword, all bits must be set to 1
_ = rd.ReadBit() // MPEG Version, set to 0 for MPEG-4 and 1 for MPEG-2
_ = rd.ReadBits(2) // Layer, always set to 0
_ = rd.ReadBit() // Protection absence, set to 1 if there is no CRC and 0 if there is CRC
objType := rd.ReadBits8(2) + 1 // Profile, the MPEG-4 Audio Object Type minus 1
sampleRateIdx := rd.ReadBits8(4) // MPEG-4 Sampling Frequency Index
_ = rd.ReadBit() // Private bit, guaranteed never to be used by MPEG, set to 0 when encoding, ignore when decoding
channels := rd.ReadBits8(3) // MPEG-4 Channel Configuration
//_ = rd.ReadBit() // Originality, set to 1 to signal originality of the audio and 0 otherwise
//_ = rd.ReadBit() // Home, set to 1 to signal home usage of the audio and 0 otherwise
//_ = rd.ReadBit() // Copyright ID bit
//_ = rd.ReadBit() // Copyright ID start
//_ = rd.ReadBits(13) // Frame length
//_ = rd.ReadBits(11) // Buffer fullness
//_ = rd.ReadBits(2) // Number of AAC frames (Raw Data Blocks) in ADTS frame minus 1
//_ = rd.ReadBits(16) // CRC check
// 3. Encode RTP config
wr := bits.NewWriter(nil)
wr.WriteBits8(objType, 5)
wr.WriteBits8(sampleRateIdx, 4)
wr.WriteBits8(channels, 4)
conf := wr.Bytes()
codec := &core.Codec{
Name: core.CodecAAC,
ClockRate: sampleRates[sampleRateIdx],
Channels: channels,
FmtpLine: FMTP + hex.EncodeToString(conf),
}
return codec
}
func ReadADTSSize(b []byte) uint16 {
// AAAAAAAA AAAABCCD EEFFFFGH HHIJKLMM MMMMMMMM MMMOOOOO OOOOOOPP (QQQQQQQQ QQQQQQQQ)
_ = b[5] // bounds
return uint16(b[3]&0b11)<<11 | uint16(b[4])<<3 | uint16(b[5]>>5)
}
func WriteADTSSize(b []byte, size uint16) {
// AAAAAAAA AAAABCCD EEFFFFGH HHIJKLMM MMMMMMMM MMMOOOOO OOOOOOPP (QQQQQQQQ QQQQQQQQ)
_ = b[5] // bounds
b[3] |= byte(size >> (8 + 3))
b[4] = byte(size >> 3)
b[5] |= byte(size << 5)
return
}
func ADTSTimeSize(b []byte) uint32 {
var units uint32
for len(b) > ADTSHeaderSize {
auSize := ReadADTSSize(b)
b = b[auSize:]
units++
}
return units * AUTime
}
func CodecToADTS(codec *core.Codec) []byte {
s := core.Between(codec.FmtpLine, "config=", ";")
conf, err := hex.DecodeString(s)
if err != nil {
return nil
}
objType, sampleFreqIdx, channels, _ := DecodeConfig(conf)
profile := objType - 1
wr := bits.NewWriter(nil)
wr.WriteAllBits(1, 12) // Syncword, all bits must be set to 1
wr.WriteBit(0) // MPEG Version, set to 0 for MPEG-4 and 1 for MPEG-2
wr.WriteBits8(0, 2) // Layer, always set to 0
wr.WriteBit(1) // Protection absence, set to 1 if there is no CRC and 0 if there is CRC
wr.WriteBits8(profile, 2) // Profile, the MPEG-4 Audio Object Type minus 1
wr.WriteBits8(sampleFreqIdx, 4) // MPEG-4 Sampling Frequency Index
wr.WriteBit(0) // Private bit, guaranteed never to be used by MPEG, set to 0 when encoding, ignore when decoding
wr.WriteBits8(channels, 3) // MPEG-4 Channel Configuration
wr.WriteBit(0) // Originality, set to 1 to signal originality of the audio and 0 otherwise
wr.WriteBit(0) // Home, set to 1 to signal home usage of the audio and 0 otherwise
wr.WriteBit(0) // Copyright ID bit
wr.WriteBit(0) // Copyright ID start
wr.WriteBits16(0, 13) // Frame length
wr.WriteAllBits(1, 11) // Buffer fullness (variable bitrate)
wr.WriteBits8(0, 2) // Number of AAC frames (Raw Data Blocks) in ADTS frame minus 1
return wr.Bytes()
}
func EncodeToADTS(codec *core.Codec, handler core.HandlerFunc) core.HandlerFunc {
adts := CodecToADTS(codec)
return func(packet *rtp.Packet) {
if !IsADTS(packet.Payload) {
b := make([]byte, ADTSHeaderSize+len(packet.Payload))
copy(b, adts)
copy(b[ADTSHeaderSize:], packet.Payload)
WriteADTSSize(b, uint16(len(b)))
clone := *packet
clone.Payload = b
handler(&clone)
} else {
handler(packet)
}
}
}
@@ -0,0 +1,59 @@
package aac
import (
"io"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/pion/rtp"
)
type Consumer struct {
core.Connection
wr *core.WriteBuffer
}
func NewConsumer() *Consumer {
medias := []*core.Media{
{
Kind: core.KindAudio,
Direction: core.DirectionSendonly,
Codecs: []*core.Codec{
{Name: core.CodecAAC},
},
},
}
wr := core.NewWriteBuffer(nil)
return &Consumer{
Connection: core.Connection{
ID: core.NewID(),
FormatName: "adts",
Medias: medias,
Transport: wr,
},
wr: wr,
}
}
func (c *Consumer) AddTrack(media *core.Media, codec *core.Codec, track *core.Receiver) error {
sender := core.NewSender(media, track.Codec)
sender.Handler = func(pkt *rtp.Packet) {
if n, err := c.wr.Write(pkt.Payload); err == nil {
c.Send += n
}
}
if track.Codec.IsRTP() {
sender.Handler = RTPToADTS(track.Codec, sender.Handler)
} else {
sender.Handler = EncodeToADTS(track.Codec, sender.Handler)
}
sender.HandleRTP(track)
c.Senders = append(c.Senders, sender)
return nil
}
func (c *Consumer) WriteTo(wr io.Writer) (int64, error) {
return c.wr.WriteTo(wr)
}
@@ -0,0 +1,85 @@
package aac
import (
"bufio"
"errors"
"io"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/pion/rtp"
)
type Producer struct {
core.Connection
rd *bufio.Reader
}
func Open(r io.Reader) (*Producer, error) {
rd := bufio.NewReader(r)
b, err := rd.Peek(ADTSHeaderSize)
if err != nil {
return nil, err
}
codec := ADTSToCodec(b)
if codec == nil {
return nil, errors.New("adts: wrong header")
}
codec.PayloadType = core.PayloadTypeRAW
medias := []*core.Media{
{
Kind: core.KindAudio,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{codec},
},
}
return &Producer{
Connection: core.Connection{
ID: core.NewID(),
FormatName: "adts",
Medias: medias,
Transport: r,
},
rd: rd,
}, nil
}
func (c *Producer) Start() error {
for {
// read ADTS header
adts := make([]byte, ADTSHeaderSize)
if _, err := io.ReadFull(c.rd, adts); err != nil {
return err
}
auSize := ReadADTSSize(adts) - ADTSHeaderSize
if HasCRC(adts) {
// skip CRC after header
if _, err := c.rd.Discard(2); err != nil {
return err
}
auSize -= 2
}
// read AAC payload after header
payload := make([]byte, auSize)
if _, err := io.ReadFull(c.rd, payload); err != nil {
return err
}
c.Recv += int(auSize)
if len(c.Receivers) == 0 {
continue
}
pkt := &rtp.Packet{
Header: rtp.Header{Timestamp: core.Now90000()},
Payload: payload,
}
c.Receivers[0].WriteRTP(pkt)
}
}
+154
View File
@@ -0,0 +1,154 @@
package aac
import (
"encoding/binary"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/pion/rtp"
)
const RTPPacketVersionAAC = 0
func RTPDepay(handler core.HandlerFunc) core.HandlerFunc {
var timestamp uint32
return func(packet *rtp.Packet) {
// support ONLY 2 bytes header size!
// streamtype=5;profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1408
// https://datatracker.ietf.org/doc/html/rfc3640
headersSize := binary.BigEndian.Uint16(packet.Payload) >> 3
//log.Printf("[RTP/AAC] units: %d, size: %4d, ts: %10d, %t", headersSize/2, len(packet.Payload), packet.Timestamp, packet.Marker)
if len(packet.Payload) < int(2+headersSize) {
// In very rare cases noname cameras may send data not according to the standard
// https://github.com/AlexxIT/go2rtc/issues/1328
if IsADTS(packet.Payload) {
clone := *packet
clone.Version = RTPPacketVersionAAC
clone.Timestamp = timestamp
clone.Payload = clone.Payload[ADTSHeaderSize:]
handler(&clone)
}
return
}
headers := packet.Payload[2 : 2+headersSize]
units := packet.Payload[2+headersSize:]
for len(headers) >= 2 {
unitSize := binary.BigEndian.Uint16(headers) >> 3
if len(units) < int(unitSize) {
return
}
unit := units[:unitSize]
headers = headers[2:]
units = units[unitSize:]
timestamp += AUTime
clone := *packet
clone.Version = RTPPacketVersionAAC
clone.Timestamp = timestamp
if IsADTS(unit) {
clone.Payload = unit[ADTSHeaderSize:]
} else {
clone.Payload = unit
}
handler(&clone)
}
}
}
func RTPPay(handler core.HandlerFunc) core.HandlerFunc {
var seq uint16
var ts uint32
return func(packet *rtp.Packet) {
if packet.Version != RTPPacketVersionAAC {
handler(packet)
return
}
// support ONLY one unit in payload
auSize := uint16(len(packet.Payload))
// 2 bytes header size + 2 bytes first payload size
payload := make([]byte, 2+2+auSize)
payload[1] = 16 // header size in bits
binary.BigEndian.PutUint16(payload[2:], auSize<<3)
copy(payload[4:], packet.Payload)
clone := rtp.Packet{
Header: rtp.Header{
Version: 2,
Marker: true,
SequenceNumber: seq,
Timestamp: ts,
},
Payload: payload,
}
handler(&clone)
seq++
ts += AUTime
}
}
func ADTStoRTP(src []byte) (dst []byte) {
dst = make([]byte, 2) // header bytes
for i, n := 0, len(src)-ADTSHeaderSize; i < n; {
auSize := ReadADTSSize(src[i:])
dst = append(dst, byte(auSize>>5), byte(auSize<<3)) // size in bits
i += int(auSize)
}
hdrSize := uint16(len(dst) - 2)
binary.BigEndian.PutUint16(dst, hdrSize<<3) // size in bits
return append(dst, src...)
}
func RTPTimeSize(b []byte) uint32 {
// convert RTP header size to units count
units := binary.BigEndian.Uint16(b) >> 4
return uint32(units) * AUTime
}
func RTPToADTS(codec *core.Codec, handler core.HandlerFunc) core.HandlerFunc {
adts := CodecToADTS(codec)
return func(packet *rtp.Packet) {
src := packet.Payload
dst := make([]byte, 0, len(src))
headersSize := binary.BigEndian.Uint16(src) >> 3
headers := src[2 : 2+headersSize]
units := src[2+headersSize:]
for len(headers) > 0 {
unitSize := binary.BigEndian.Uint16(headers) >> 3
headers = headers[2:]
unit := units[:unitSize]
units = units[unitSize:]
if !IsADTS(unit) {
i := len(dst)
dst = append(dst, adts...)
WriteADTSSize(dst[i:], ADTSHeaderSize+uint16(len(unit)))
}
dst = append(dst, unit...)
}
clone := *packet
clone.Version = RTPPacketVersionAAC
clone.Payload = dst
handler(&clone)
}
}
func RTPToCodec(b []byte) *core.Codec {
hdrSize := binary.BigEndian.Uint16(b) / 8
return ADTSToCodec(b[2+hdrSize:])
}
@@ -0,0 +1,33 @@
package aac
import (
"encoding/hex"
"testing"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/pion/rtp"
"github.com/stretchr/testify/require"
)
func TestBuggy_RTSP_AAC(t *testing.T) {
// https: //github.com/AlexxIT/go2rtc/issues/1328
payload, _ := hex.DecodeString("fff16080431ffc211ad4458aa309a1c0a8761a230502b7c74b2b5499252a010555e32e460128303c8ace4fd3260d654a424f7e7c65eddc96735fc6f1ac0edf94fdefa0e0bd6370da1c07b9c0e77a9d6e86b196a1ac7439dcafadcffcf6d89f60ac67f8884868e931383ad3e40cf5495470d1f606ef6f7624d285b951ebfa0e42641ab98f1371182b237d14f1bd16ad714fa2f1c6a7d23ebde7a0e34a2eca156a608a4caec49d9dca4b6fe2a09e9cdbf762c5b4148a3914abb7959c991228b0837b5988334b9fc18b8fac689b5ca1e4661573bbb8b253a86cae7ec14ace49969a9a76fd571ab6e650764cb59114d61dcedf07ac61b39e4ac66adebfd0d0ab45d518dd3c161049823f150864d977cf0855172ac8482e4b25fe911325d19617558c5405af74aff5492e4599bee53f2dbdf0503730af37078550f84c956b7ee89aae83c154fa2fa6e6792c5ddd5cd5cf6bb96bf055fee7f93bed59ffb039daee5ea7e5593cb194e9091e417c67d8f73026a6a6ae056e808f7c65c03d1b9197d3709ceb63bc7b979f7ba71df5e7c6395d99d6ea229000a6bc16fb4346d6b27d32f5d8d1200736d9366d59c0c9547210813b602473da9c46f9015bbb37594c1dd90cd6a36e96bd5d6a1445ab93c9e65505ec2c722bb4cc27a10600139a48c83594dde145253c386f6627d8c6e5102fe3828a590c709bc87f55b37e97d1ae72b017b09c6bb2c13299817bb45cc67318e10b6822075b97c6a03ec1c0")
packet := &rtp.Packet{
Header: rtp.Header{
Version: 2,
Marker: true,
SequenceNumber: 36944,
Timestamp: 4217191328,
SSRC: 12892774,
},
Payload: payload,
}
var size int
RTPDepay(func(packet *core.Packet) {
size = len(packet.Payload)
})(packet)
require.Equal(t, len(payload), size+ADTSHeaderSize)
}
@@ -0,0 +1,23 @@
## Build
```shell
x86_64-linux-gnu-gcc -w -static asound_arch.c -o asound_amd64
i686-linux-gnu-gcc -w -static asound_arch.c -o asound_i386
aarch64-linux-gnu-gcc -w -static asound_arch.c -o asound_arm64
arm-linux-gnueabihf-gcc -w -static asound_arch.c -o asound_arm
mipsel-linux-gnu-gcc -w -static asound_arch.c -o asound_mipsle -D_TIME_BITS=32
```
## Useful links
- https://github.com/torvalds/linux/blob/master/include/uapi/sound/asound.h
- https://github.com/yobert/alsa
- https://github.com/Narsil/alsa-go
- https://github.com/alsa-project/alsa-lib
- https://github.com/anisse/alsa
- https://github.com/tinyalsa/tinyalsa
**Broken pipe**
- https://stackoverflow.com/questions/26545139/alsa-cannot-recovery-from-underrun-prepare-failed-broken-pipe
- https://klipspringer.avadeaux.net/alsa-broken-pipe-errors/
@@ -0,0 +1,90 @@
package alsa
import (
"github.com/AlexxIT/go2rtc/pkg/alsa/device"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/pcm"
"github.com/pion/rtp"
)
type Capture struct {
core.Connection
dev *device.Device
closed core.Waiter
}
func newCapture(dev *device.Device) (*Capture, error) {
medias := []*core.Media{
{
Kind: core.KindAudio,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{
{Name: core.CodecPCML, ClockRate: 16000},
},
},
}
return &Capture{
Connection: core.Connection{
ID: core.NewID(),
FormatName: "alsa",
Medias: medias,
Transport: dev,
},
dev: dev,
}, nil
}
func (c *Capture) Start() error {
dst := c.Medias[0].Codecs[0]
src := &core.Codec{
Name: dst.Name,
ClockRate: c.dev.GetRateNear(dst.ClockRate),
Channels: c.dev.GetChannelsNear(dst.Channels),
}
if err := c.dev.SetHWParams(device.SNDRV_PCM_FORMAT_S16_LE, src.ClockRate, src.Channels); err != nil {
return err
}
transcode := transcodeFunc(dst, src)
frameBytes := int(pcm.BytesPerFrame(src))
var ts uint32
// readBufferSize for 20ms interval
readBufferSize := 20 * frameBytes * int(src.ClockRate) / 1000
b := make([]byte, readBufferSize)
for {
n, err := c.dev.Read(b)
if err != nil {
return err
}
c.Recv += n
if len(c.Receivers) == 0 {
continue
}
pkt := &rtp.Packet{
Header: rtp.Header{
Version: 2,
Marker: true,
Timestamp: ts,
},
Payload: transcode(b[:n]),
}
c.Receivers[0].WriteRTP(pkt)
ts += uint32(n / frameBytes)
}
}
func transcodeFunc(dst, src *core.Codec) func([]byte) []byte {
if dst.ClockRate == src.ClockRate && dst.Channels == src.Channels {
return func(b []byte) []byte {
return b
}
}
return pcm.Transcode(dst, src)
}
@@ -0,0 +1,148 @@
//go:build 386 || arm
package device
type unsigned_char = byte
type signed_int = int32
type unsigned_int = uint32
type signed_long = int64
type unsigned_long = uint64
type __u32 = uint32
type void__user = uintptr
const (
SNDRV_PCM_STREAM_PLAYBACK = 0
SNDRV_PCM_STREAM_CAPTURE = 1
SNDRV_PCM_ACCESS_MMAP_INTERLEAVED = 0
SNDRV_PCM_ACCESS_MMAP_NONINTERLEAVED = 1
SNDRV_PCM_ACCESS_MMAP_COMPLEX = 2
SNDRV_PCM_ACCESS_RW_INTERLEAVED = 3
SNDRV_PCM_ACCESS_RW_NONINTERLEAVED = 4
SNDRV_PCM_FORMAT_S8 = 0
SNDRV_PCM_FORMAT_U8 = 1
SNDRV_PCM_FORMAT_S16_LE = 2
SNDRV_PCM_FORMAT_S16_BE = 3
SNDRV_PCM_FORMAT_U16_LE = 4
SNDRV_PCM_FORMAT_U16_BE = 5
SNDRV_PCM_FORMAT_S24_LE = 6
SNDRV_PCM_FORMAT_S24_BE = 7
SNDRV_PCM_FORMAT_U24_LE = 8
SNDRV_PCM_FORMAT_U24_BE = 9
SNDRV_PCM_FORMAT_S32_LE = 10
SNDRV_PCM_FORMAT_S32_BE = 11
SNDRV_PCM_FORMAT_U32_LE = 12
SNDRV_PCM_FORMAT_U32_BE = 13
SNDRV_PCM_FORMAT_FLOAT_LE = 14
SNDRV_PCM_FORMAT_FLOAT_BE = 15
SNDRV_PCM_FORMAT_FLOAT64_LE = 16
SNDRV_PCM_FORMAT_FLOAT64_BE = 17
SNDRV_PCM_FORMAT_MU_LAW = 20
SNDRV_PCM_FORMAT_A_LAW = 21
SNDRV_PCM_FORMAT_MPEG = 23
SNDRV_PCM_IOCTL_PVERSION = 0x80044100
SNDRV_PCM_IOCTL_INFO = 0x81204101
SNDRV_PCM_IOCTL_HW_REFINE = 0xc25c4110
SNDRV_PCM_IOCTL_HW_PARAMS = 0xc25c4111
SNDRV_PCM_IOCTL_SW_PARAMS = 0xc0684113
SNDRV_PCM_IOCTL_PREPARE = 0x00004140
SNDRV_PCM_IOCTL_WRITEI_FRAMES = 0x400c4150
SNDRV_PCM_IOCTL_READI_FRAMES = 0x800c4151
)
type snd_pcm_info struct { // size 288
device unsigned_int // offset 0, size 4
subdevice unsigned_int // offset 4, size 4
stream signed_int // offset 8, size 4
card signed_int // offset 12, size 4
id [64]unsigned_char // offset 16, size 64
name [80]unsigned_char // offset 80, size 80
subname [32]unsigned_char // offset 160, size 32
dev_class signed_int // offset 192, size 4
dev_subclass signed_int // offset 196, size 4
subdevices_count unsigned_int // offset 200, size 4
subdevices_avail unsigned_int // offset 204, size 4
pad1 [16]unsigned_char
reserved [64]unsigned_char // offset 224, size 64
}
type snd_pcm_uframes_t = unsigned_long
type snd_pcm_sframes_t = signed_long
type snd_xferi struct { // size 12
result snd_pcm_sframes_t // offset 0, size 4
buf void__user // offset 4, size 4
frames snd_pcm_uframes_t // offset 8, size 4
}
const (
SNDRV_PCM_HW_PARAM_ACCESS = 0
SNDRV_PCM_HW_PARAM_FORMAT = 1
SNDRV_PCM_HW_PARAM_SUBFORMAT = 2
SNDRV_PCM_HW_PARAM_FIRST_MASK = 0
SNDRV_PCM_HW_PARAM_LAST_MASK = 2
SNDRV_PCM_HW_PARAM_SAMPLE_BITS = 8
SNDRV_PCM_HW_PARAM_FRAME_BITS = 9
SNDRV_PCM_HW_PARAM_CHANNELS = 10
SNDRV_PCM_HW_PARAM_RATE = 11
SNDRV_PCM_HW_PARAM_PERIOD_TIME = 12
SNDRV_PCM_HW_PARAM_PERIOD_SIZE = 13
SNDRV_PCM_HW_PARAM_PERIOD_BYTES = 14
SNDRV_PCM_HW_PARAM_PERIODS = 15
SNDRV_PCM_HW_PARAM_BUFFER_TIME = 16
SNDRV_PCM_HW_PARAM_BUFFER_SIZE = 17
SNDRV_PCM_HW_PARAM_BUFFER_BYTES = 18
SNDRV_PCM_HW_PARAM_TICK_TIME = 19
SNDRV_PCM_HW_PARAM_FIRST_INTERVAL = 8
SNDRV_PCM_HW_PARAM_LAST_INTERVAL = 19
SNDRV_MASK_MAX = 256
SNDRV_PCM_TSTAMP_NONE = 0
SNDRV_PCM_TSTAMP_ENABLE = 1
)
type snd_mask struct { // size 32
bits [(SNDRV_MASK_MAX + 31) / 32]__u32 // offset 0, size 32
}
type snd_interval struct { // size 12
min unsigned_int // offset 0, size 4
max unsigned_int // offset 4, size 4
bit unsigned_int
}
type snd_pcm_hw_params struct { // size 604
flags unsigned_int // offset 0, size 4
masks [SNDRV_PCM_HW_PARAM_LAST_MASK - SNDRV_PCM_HW_PARAM_FIRST_MASK + 1]snd_mask // offset 4, size 96
mres [5]snd_mask // offset 100, size 160
intervals [SNDRV_PCM_HW_PARAM_LAST_INTERVAL - SNDRV_PCM_HW_PARAM_FIRST_INTERVAL + 1]snd_interval // offset 260, size 144
ires [9]snd_interval // offset 404, size 108
rmask unsigned_int // offset 512, size 4
cmask unsigned_int // offset 516, size 4
info unsigned_int // offset 520, size 4
msbits unsigned_int // offset 524, size 4
rate_num unsigned_int // offset 528, size 4
rate_den unsigned_int // offset 532, size 4
fifo_size snd_pcm_uframes_t // offset 536, size 4
reserved [64]unsigned_char // offset 540, size 64
}
type snd_pcm_sw_params struct { // size 104
tstamp_mode signed_int // offset 0, size 4
period_step unsigned_int // offset 4, size 4
sleep_min unsigned_int // offset 8, size 4
avail_min snd_pcm_uframes_t // offset 12, size 4
xfer_align snd_pcm_uframes_t // offset 16, size 4
start_threshold snd_pcm_uframes_t // offset 20, size 4
stop_threshold snd_pcm_uframes_t // offset 24, size 4
silence_threshold snd_pcm_uframes_t // offset 28, size 4
silence_size snd_pcm_uframes_t // offset 32, size 4
boundary snd_pcm_uframes_t // offset 36, size 4
proto unsigned_int // offset 40, size 4
tstamp_type unsigned_int // offset 44, size 4
reserved [56]unsigned_char // offset 48, size 56
}
@@ -0,0 +1,148 @@
//go:build amd64 || arm64
package device
type unsigned_char = byte
type signed_int = int32
type unsigned_int = uint32
type signed_long = int64
type unsigned_long = uint64
type __u32 = uint32
type void__user = uintptr
const (
SNDRV_PCM_STREAM_PLAYBACK = 0
SNDRV_PCM_STREAM_CAPTURE = 1
SNDRV_PCM_ACCESS_MMAP_INTERLEAVED = 0
SNDRV_PCM_ACCESS_MMAP_NONINTERLEAVED = 1
SNDRV_PCM_ACCESS_MMAP_COMPLEX = 2
SNDRV_PCM_ACCESS_RW_INTERLEAVED = 3
SNDRV_PCM_ACCESS_RW_NONINTERLEAVED = 4
SNDRV_PCM_FORMAT_S8 = 0
SNDRV_PCM_FORMAT_U8 = 1
SNDRV_PCM_FORMAT_S16_LE = 2
SNDRV_PCM_FORMAT_S16_BE = 3
SNDRV_PCM_FORMAT_U16_LE = 4
SNDRV_PCM_FORMAT_U16_BE = 5
SNDRV_PCM_FORMAT_S24_LE = 6
SNDRV_PCM_FORMAT_S24_BE = 7
SNDRV_PCM_FORMAT_U24_LE = 8
SNDRV_PCM_FORMAT_U24_BE = 9
SNDRV_PCM_FORMAT_S32_LE = 10
SNDRV_PCM_FORMAT_S32_BE = 11
SNDRV_PCM_FORMAT_U32_LE = 12
SNDRV_PCM_FORMAT_U32_BE = 13
SNDRV_PCM_FORMAT_FLOAT_LE = 14
SNDRV_PCM_FORMAT_FLOAT_BE = 15
SNDRV_PCM_FORMAT_FLOAT64_LE = 16
SNDRV_PCM_FORMAT_FLOAT64_BE = 17
SNDRV_PCM_FORMAT_MU_LAW = 20
SNDRV_PCM_FORMAT_A_LAW = 21
SNDRV_PCM_FORMAT_MPEG = 23
SNDRV_PCM_IOCTL_PVERSION = 0x80044100
SNDRV_PCM_IOCTL_INFO = 0x81204101
SNDRV_PCM_IOCTL_HW_REFINE = 0xc2604110
SNDRV_PCM_IOCTL_HW_PARAMS = 0xc2604111
SNDRV_PCM_IOCTL_SW_PARAMS = 0xc0884113
SNDRV_PCM_IOCTL_PREPARE = 0x00004140
SNDRV_PCM_IOCTL_WRITEI_FRAMES = 0x40184150
SNDRV_PCM_IOCTL_READI_FRAMES = 0x80184151
)
type snd_pcm_info struct { // size 288
device unsigned_int // offset 0, size 4
subdevice unsigned_int // offset 4, size 4
stream signed_int // offset 8, size 4
card signed_int // offset 12, size 4
id [64]unsigned_char // offset 16, size 64
name [80]unsigned_char // offset 80, size 80
subname [32]unsigned_char // offset 160, size 32
dev_class signed_int // offset 192, size 4
dev_subclass signed_int // offset 196, size 4
subdevices_count unsigned_int // offset 200, size 4
subdevices_avail unsigned_int // offset 204, size 4
pad1 [16]unsigned_char
reserved [64]unsigned_char // offset 224, size 64
}
type snd_pcm_uframes_t = unsigned_long
type snd_pcm_sframes_t = signed_long
type snd_xferi struct { // size 24
result snd_pcm_sframes_t // offset 0, size 8
buf void__user // offset 8, size 8
frames snd_pcm_uframes_t // offset 16, size 8
}
const (
SNDRV_PCM_HW_PARAM_ACCESS = 0
SNDRV_PCM_HW_PARAM_FORMAT = 1
SNDRV_PCM_HW_PARAM_SUBFORMAT = 2
SNDRV_PCM_HW_PARAM_FIRST_MASK = 0
SNDRV_PCM_HW_PARAM_LAST_MASK = 2
SNDRV_PCM_HW_PARAM_SAMPLE_BITS = 8
SNDRV_PCM_HW_PARAM_FRAME_BITS = 9
SNDRV_PCM_HW_PARAM_CHANNELS = 10
SNDRV_PCM_HW_PARAM_RATE = 11
SNDRV_PCM_HW_PARAM_PERIOD_TIME = 12
SNDRV_PCM_HW_PARAM_PERIOD_SIZE = 13
SNDRV_PCM_HW_PARAM_PERIOD_BYTES = 14
SNDRV_PCM_HW_PARAM_PERIODS = 15
SNDRV_PCM_HW_PARAM_BUFFER_TIME = 16
SNDRV_PCM_HW_PARAM_BUFFER_SIZE = 17
SNDRV_PCM_HW_PARAM_BUFFER_BYTES = 18
SNDRV_PCM_HW_PARAM_TICK_TIME = 19
SNDRV_PCM_HW_PARAM_FIRST_INTERVAL = 8
SNDRV_PCM_HW_PARAM_LAST_INTERVAL = 19
SNDRV_MASK_MAX = 256
SNDRV_PCM_TSTAMP_NONE = 0
SNDRV_PCM_TSTAMP_ENABLE = 1
)
type snd_mask struct { // size 32
bits [(SNDRV_MASK_MAX + 31) / 32]__u32 // offset 0, size 32
}
type snd_interval struct { // size 12
min unsigned_int // offset 0, size 4
max unsigned_int // offset 4, size 4
bit unsigned_int
}
type snd_pcm_hw_params struct { // size 608
flags unsigned_int // offset 0, size 4
masks [SNDRV_PCM_HW_PARAM_LAST_MASK - SNDRV_PCM_HW_PARAM_FIRST_MASK + 1]snd_mask // offset 4, size 96
mres [5]snd_mask // offset 100, size 160
intervals [SNDRV_PCM_HW_PARAM_LAST_INTERVAL - SNDRV_PCM_HW_PARAM_FIRST_INTERVAL + 1]snd_interval // offset 260, size 144
ires [9]snd_interval // offset 404, size 108
rmask unsigned_int // offset 512, size 4
cmask unsigned_int // offset 516, size 4
info unsigned_int // offset 520, size 4
msbits unsigned_int // offset 524, size 4
rate_num unsigned_int // offset 528, size 4
rate_den unsigned_int // offset 532, size 4
fifo_size snd_pcm_uframes_t // offset 536, size 8
reserved [64]unsigned_char // offset 544, size 64
}
type snd_pcm_sw_params struct { // size 136
tstamp_mode signed_int // offset 0, size 4
period_step unsigned_int // offset 4, size 4
sleep_min unsigned_int // offset 8, size 4
avail_min snd_pcm_uframes_t // offset 16, size 8
xfer_align snd_pcm_uframes_t // offset 24, size 8
start_threshold snd_pcm_uframes_t // offset 32, size 8
stop_threshold snd_pcm_uframes_t // offset 40, size 8
silence_threshold snd_pcm_uframes_t // offset 48, size 8
silence_size snd_pcm_uframes_t // offset 56, size 8
boundary snd_pcm_uframes_t // offset 64, size 8
proto unsigned_int // offset 72, size 4
tstamp_type unsigned_int // offset 76, size 4
reserved [56]unsigned_char // offset 80, size 56
}
@@ -0,0 +1,164 @@
//go:build ignore
#include <stdio.h>
#include <stddef.h>
#include <sys/ioctl.h>
#include <sound/asound.h>
#define print_line(text) printf("%s\n", text)
#define print_hex_const(name) printf("\t%s = 0x%08lx\n", #name, name)
#define print_int_const(con) printf("\t%s = %d\n", #con, con)
#define print_struct_header(str) printf("type %s struct { // size %lu\n", #str, sizeof(struct str))
#define print_struct_member(str, mem, typ) printf("\t%s %s // offset %lu, size %lu\n", #mem == "type" ? "typ" : #mem, typ, offsetof(struct str, mem), sizeof((struct str){0}.mem))
// https://github.com/torvalds/linux/blob/master/include/uapi/sound/asound.h
int main() {
print_line("package device\n");
print_line("type unsigned_char = byte");
print_line("type signed_int = int32");
print_line("type unsigned_int = uint32");
print_line("type signed_long = int64");
print_line("type unsigned_long = uint64");
print_line("type __u32 = uint32");
print_line("type void__user = uintptr\n");
print_line("const (");
print_int_const(SNDRV_PCM_STREAM_PLAYBACK);
print_int_const(SNDRV_PCM_STREAM_CAPTURE);
print_line("");
print_int_const(SNDRV_PCM_ACCESS_MMAP_INTERLEAVED);
print_int_const(SNDRV_PCM_ACCESS_MMAP_NONINTERLEAVED);
print_int_const(SNDRV_PCM_ACCESS_MMAP_COMPLEX);
print_int_const(SNDRV_PCM_ACCESS_RW_INTERLEAVED);
print_int_const(SNDRV_PCM_ACCESS_RW_NONINTERLEAVED);
print_line("");
print_int_const(SNDRV_PCM_FORMAT_S8);
print_int_const(SNDRV_PCM_FORMAT_U8);
print_int_const(SNDRV_PCM_FORMAT_S16_LE);
print_int_const(SNDRV_PCM_FORMAT_S16_BE);
print_int_const(SNDRV_PCM_FORMAT_U16_LE);
print_int_const(SNDRV_PCM_FORMAT_U16_BE);
print_int_const(SNDRV_PCM_FORMAT_S24_LE);
print_int_const(SNDRV_PCM_FORMAT_S24_BE);
print_int_const(SNDRV_PCM_FORMAT_U24_LE);
print_int_const(SNDRV_PCM_FORMAT_U24_BE);
print_int_const(SNDRV_PCM_FORMAT_S32_LE);
print_int_const(SNDRV_PCM_FORMAT_S32_BE);
print_int_const(SNDRV_PCM_FORMAT_U32_LE);
print_int_const(SNDRV_PCM_FORMAT_U32_BE);
print_int_const(SNDRV_PCM_FORMAT_FLOAT_LE);
print_int_const(SNDRV_PCM_FORMAT_FLOAT_BE);
print_int_const(SNDRV_PCM_FORMAT_FLOAT64_LE);
print_int_const(SNDRV_PCM_FORMAT_FLOAT64_BE);
print_int_const(SNDRV_PCM_FORMAT_MU_LAW);
print_int_const(SNDRV_PCM_FORMAT_A_LAW);
print_int_const(SNDRV_PCM_FORMAT_MPEG);
print_line("");
print_hex_const(SNDRV_PCM_IOCTL_PVERSION); // A 0x00
print_hex_const(SNDRV_PCM_IOCTL_INFO); // A 0x01
print_hex_const(SNDRV_PCM_IOCTL_HW_REFINE); // A 0x10
print_hex_const(SNDRV_PCM_IOCTL_HW_PARAMS); // A 0x11
print_hex_const(SNDRV_PCM_IOCTL_SW_PARAMS); // A 0x13
print_hex_const(SNDRV_PCM_IOCTL_PREPARE); // A 0x40
print_hex_const(SNDRV_PCM_IOCTL_WRITEI_FRAMES); // A 0x50
print_hex_const(SNDRV_PCM_IOCTL_READI_FRAMES); // A 0x51
print_line(")\n");
print_struct_header(snd_pcm_info);
print_struct_member(snd_pcm_info, device, "unsigned_int");
print_struct_member(snd_pcm_info, subdevice, "unsigned_int");
print_struct_member(snd_pcm_info, stream, "signed_int");
print_struct_member(snd_pcm_info, card, "signed_int");
print_struct_member(snd_pcm_info, id, "[64]unsigned_char");
print_struct_member(snd_pcm_info, name, "[80]unsigned_char");
print_struct_member(snd_pcm_info, subname, "[32]unsigned_char");
print_struct_member(snd_pcm_info, dev_class, "signed_int");
print_struct_member(snd_pcm_info, dev_subclass, "signed_int");
print_struct_member(snd_pcm_info, subdevices_count, "unsigned_int");
print_struct_member(snd_pcm_info, subdevices_avail, "unsigned_int");
print_line("\tpad1 [16]unsigned_char");
print_struct_member(snd_pcm_info, reserved, "[64]unsigned_char");
print_line("}\n");
print_line("type snd_pcm_uframes_t = unsigned_long");
print_line("type snd_pcm_sframes_t = signed_long\n");
print_struct_header(snd_xferi);
print_struct_member(snd_xferi, result, "snd_pcm_sframes_t");
print_struct_member(snd_xferi, buf, "void__user");
print_struct_member(snd_xferi, frames, "snd_pcm_uframes_t");
print_line("}\n");
print_line("const (");
print_int_const(SNDRV_PCM_HW_PARAM_ACCESS);
print_int_const(SNDRV_PCM_HW_PARAM_FORMAT);
print_int_const(SNDRV_PCM_HW_PARAM_SUBFORMAT);
print_int_const(SNDRV_PCM_HW_PARAM_FIRST_MASK);
print_int_const(SNDRV_PCM_HW_PARAM_LAST_MASK);
print_line("");
print_int_const(SNDRV_PCM_HW_PARAM_SAMPLE_BITS);
print_int_const(SNDRV_PCM_HW_PARAM_FRAME_BITS);
print_int_const(SNDRV_PCM_HW_PARAM_CHANNELS);
print_int_const(SNDRV_PCM_HW_PARAM_RATE);
print_int_const(SNDRV_PCM_HW_PARAM_PERIOD_TIME);
print_int_const(SNDRV_PCM_HW_PARAM_PERIOD_SIZE);
print_int_const(SNDRV_PCM_HW_PARAM_PERIOD_BYTES);
print_int_const(SNDRV_PCM_HW_PARAM_PERIODS);
print_int_const(SNDRV_PCM_HW_PARAM_BUFFER_TIME);
print_int_const(SNDRV_PCM_HW_PARAM_BUFFER_SIZE);
print_int_const(SNDRV_PCM_HW_PARAM_BUFFER_BYTES);
print_int_const(SNDRV_PCM_HW_PARAM_TICK_TIME);
print_int_const(SNDRV_PCM_HW_PARAM_FIRST_INTERVAL);
print_int_const(SNDRV_PCM_HW_PARAM_LAST_INTERVAL);
print_line("");
print_int_const(SNDRV_MASK_MAX);
print_line("");
print_int_const(SNDRV_PCM_TSTAMP_NONE);
print_int_const(SNDRV_PCM_TSTAMP_ENABLE);
print_line(")\n");
print_struct_header(snd_mask);
print_struct_member(snd_mask, bits, "[(SNDRV_MASK_MAX+31)/32]__u32");
print_line("}\n");
print_struct_header(snd_interval);
print_struct_member(snd_interval, min, "unsigned_int");
print_struct_member(snd_interval, max, "unsigned_int");
print_line("\tbit unsigned_int");
print_line("}\n");
print_struct_header(snd_pcm_hw_params);
print_struct_member(snd_pcm_hw_params, flags, "unsigned_int");
print_struct_member(snd_pcm_hw_params, masks, "[SNDRV_PCM_HW_PARAM_LAST_MASK-SNDRV_PCM_HW_PARAM_FIRST_MASK+1]snd_mask");
print_struct_member(snd_pcm_hw_params, mres, "[5]snd_mask");
print_struct_member(snd_pcm_hw_params, intervals, "[SNDRV_PCM_HW_PARAM_LAST_INTERVAL-SNDRV_PCM_HW_PARAM_FIRST_INTERVAL+1]snd_interval");
print_struct_member(snd_pcm_hw_params, ires, "[9]snd_interval");
print_struct_member(snd_pcm_hw_params, rmask, "unsigned_int");
print_struct_member(snd_pcm_hw_params, cmask, "unsigned_int");
print_struct_member(snd_pcm_hw_params, info, "unsigned_int");
print_struct_member(snd_pcm_hw_params, msbits, "unsigned_int");
print_struct_member(snd_pcm_hw_params, rate_num, "unsigned_int");
print_struct_member(snd_pcm_hw_params, rate_den, "unsigned_int");
print_struct_member(snd_pcm_hw_params, fifo_size, "snd_pcm_uframes_t");
print_struct_member(snd_pcm_hw_params, reserved, "[64]unsigned_char");
print_line("}\n");
print_struct_header(snd_pcm_sw_params);
print_struct_member(snd_pcm_sw_params, tstamp_mode, "signed_int");
print_struct_member(snd_pcm_sw_params, period_step, "unsigned_int");
print_struct_member(snd_pcm_sw_params, sleep_min, "unsigned_int");
print_struct_member(snd_pcm_sw_params, avail_min, "snd_pcm_uframes_t");
print_struct_member(snd_pcm_sw_params, xfer_align, "snd_pcm_uframes_t");
print_struct_member(snd_pcm_sw_params, start_threshold, "snd_pcm_uframes_t");
print_struct_member(snd_pcm_sw_params, stop_threshold, "snd_pcm_uframes_t");
print_struct_member(snd_pcm_sw_params, silence_threshold, "snd_pcm_uframes_t");
print_struct_member(snd_pcm_sw_params, silence_size, "snd_pcm_uframes_t");
print_struct_member(snd_pcm_sw_params, boundary, "snd_pcm_uframes_t");
print_struct_member(snd_pcm_sw_params, proto, "unsigned_int");
print_struct_member(snd_pcm_sw_params, tstamp_type, "unsigned_int");
print_struct_member(snd_pcm_sw_params, reserved, "[56]unsigned_char");
print_line("}\n");
return 0;
}
@@ -0,0 +1,146 @@
package device
type unsigned_char = byte
type signed_int = int32
type unsigned_int = uint32
type signed_long = int64
type unsigned_long = uint64
type __u32 = uint32
type void__user = uintptr
const (
SNDRV_PCM_STREAM_PLAYBACK = 0
SNDRV_PCM_STREAM_CAPTURE = 1
SNDRV_PCM_ACCESS_MMAP_INTERLEAVED = 0
SNDRV_PCM_ACCESS_MMAP_NONINTERLEAVED = 1
SNDRV_PCM_ACCESS_MMAP_COMPLEX = 2
SNDRV_PCM_ACCESS_RW_INTERLEAVED = 3
SNDRV_PCM_ACCESS_RW_NONINTERLEAVED = 4
SNDRV_PCM_FORMAT_S8 = 0
SNDRV_PCM_FORMAT_U8 = 1
SNDRV_PCM_FORMAT_S16_LE = 2
SNDRV_PCM_FORMAT_S16_BE = 3
SNDRV_PCM_FORMAT_U16_LE = 4
SNDRV_PCM_FORMAT_U16_BE = 5
SNDRV_PCM_FORMAT_S24_LE = 6
SNDRV_PCM_FORMAT_S24_BE = 7
SNDRV_PCM_FORMAT_U24_LE = 8
SNDRV_PCM_FORMAT_U24_BE = 9
SNDRV_PCM_FORMAT_S32_LE = 10
SNDRV_PCM_FORMAT_S32_BE = 11
SNDRV_PCM_FORMAT_U32_LE = 12
SNDRV_PCM_FORMAT_U32_BE = 13
SNDRV_PCM_FORMAT_FLOAT_LE = 14
SNDRV_PCM_FORMAT_FLOAT_BE = 15
SNDRV_PCM_FORMAT_FLOAT64_LE = 16
SNDRV_PCM_FORMAT_FLOAT64_BE = 17
SNDRV_PCM_FORMAT_MU_LAW = 20
SNDRV_PCM_FORMAT_A_LAW = 21
SNDRV_PCM_FORMAT_MPEG = 23
SNDRV_PCM_IOCTL_PVERSION = 0x40044100
SNDRV_PCM_IOCTL_INFO = 0x41204101
SNDRV_PCM_IOCTL_HW_REFINE = 0xc25c4110
SNDRV_PCM_IOCTL_HW_PARAMS = 0xc25c4111
SNDRV_PCM_IOCTL_SW_PARAMS = 0xc0684113
SNDRV_PCM_IOCTL_PREPARE = 0x20004140
SNDRV_PCM_IOCTL_WRITEI_FRAMES = 0x800c4150
SNDRV_PCM_IOCTL_READI_FRAMES = 0x400c4151
)
type snd_pcm_info struct { // size 288
device unsigned_int // offset 0, size 4
subdevice unsigned_int // offset 4, size 4
stream signed_int // offset 8, size 4
card signed_int // offset 12, size 4
id [64]unsigned_char // offset 16, size 64
name [80]unsigned_char // offset 80, size 80
subname [32]unsigned_char // offset 160, size 32
dev_class signed_int // offset 192, size 4
dev_subclass signed_int // offset 196, size 4
subdevices_count unsigned_int // offset 200, size 4
subdevices_avail unsigned_int // offset 204, size 4
pad1 [16]unsigned_char
reserved [64]unsigned_char // offset 224, size 64
}
type snd_pcm_uframes_t = unsigned_long
type snd_pcm_sframes_t = signed_long
type snd_xferi struct { // size 12
result snd_pcm_sframes_t // offset 0, size 4
buf void__user // offset 4, size 4
frames snd_pcm_uframes_t // offset 8, size 4
}
const (
SNDRV_PCM_HW_PARAM_ACCESS = 0
SNDRV_PCM_HW_PARAM_FORMAT = 1
SNDRV_PCM_HW_PARAM_SUBFORMAT = 2
SNDRV_PCM_HW_PARAM_FIRST_MASK = 0
SNDRV_PCM_HW_PARAM_LAST_MASK = 2
SNDRV_PCM_HW_PARAM_SAMPLE_BITS = 8
SNDRV_PCM_HW_PARAM_FRAME_BITS = 9
SNDRV_PCM_HW_PARAM_CHANNELS = 10
SNDRV_PCM_HW_PARAM_RATE = 11
SNDRV_PCM_HW_PARAM_PERIOD_TIME = 12
SNDRV_PCM_HW_PARAM_PERIOD_SIZE = 13
SNDRV_PCM_HW_PARAM_PERIOD_BYTES = 14
SNDRV_PCM_HW_PARAM_PERIODS = 15
SNDRV_PCM_HW_PARAM_BUFFER_TIME = 16
SNDRV_PCM_HW_PARAM_BUFFER_SIZE = 17
SNDRV_PCM_HW_PARAM_BUFFER_BYTES = 18
SNDRV_PCM_HW_PARAM_TICK_TIME = 19
SNDRV_PCM_HW_PARAM_FIRST_INTERVAL = 8
SNDRV_PCM_HW_PARAM_LAST_INTERVAL = 19
SNDRV_MASK_MAX = 256
SNDRV_PCM_TSTAMP_NONE = 0
SNDRV_PCM_TSTAMP_ENABLE = 1
)
type snd_mask struct { // size 32
bits [(SNDRV_MASK_MAX + 31) / 32]__u32 // offset 0, size 32
}
type snd_interval struct { // size 12
min unsigned_int // offset 0, size 4
max unsigned_int // offset 4, size 4
bit unsigned_int
}
type snd_pcm_hw_params struct { // size 604
flags unsigned_int // offset 0, size 4
masks [SNDRV_PCM_HW_PARAM_LAST_MASK - SNDRV_PCM_HW_PARAM_FIRST_MASK + 1]snd_mask // offset 4, size 96
mres [5]snd_mask // offset 100, size 160
intervals [SNDRV_PCM_HW_PARAM_LAST_INTERVAL - SNDRV_PCM_HW_PARAM_FIRST_INTERVAL + 1]snd_interval // offset 260, size 144
ires [9]snd_interval // offset 404, size 108
rmask unsigned_int // offset 512, size 4
cmask unsigned_int // offset 516, size 4
info unsigned_int // offset 520, size 4
msbits unsigned_int // offset 524, size 4
rate_num unsigned_int // offset 528, size 4
rate_den unsigned_int // offset 532, size 4
fifo_size snd_pcm_uframes_t // offset 536, size 4
reserved [64]unsigned_char // offset 540, size 64
}
type snd_pcm_sw_params struct { // size 104
tstamp_mode signed_int // offset 0, size 4
period_step unsigned_int // offset 4, size 4
sleep_min unsigned_int // offset 8, size 4
avail_min snd_pcm_uframes_t // offset 12, size 4
xfer_align snd_pcm_uframes_t // offset 16, size 4
start_threshold snd_pcm_uframes_t // offset 20, size 4
stop_threshold snd_pcm_uframes_t // offset 24, size 4
silence_threshold snd_pcm_uframes_t // offset 28, size 4
silence_size snd_pcm_uframes_t // offset 32, size 4
boundary snd_pcm_uframes_t // offset 36, size 4
proto unsigned_int // offset 40, size 4
tstamp_type unsigned_int // offset 44, size 4
reserved [56]unsigned_char // offset 48, size 56
}
@@ -0,0 +1,231 @@
package device
import (
"fmt"
"syscall"
"unsafe"
)
type Device struct {
fd uintptr
path string
hwparams snd_pcm_hw_params
frameBytes int // sample size * channels
}
func Open(path string) (*Device, error) {
// important to use nonblock because can get lock
fd, err := syscall.Open(path, syscall.O_RDWR|syscall.O_NONBLOCK, 0)
if err != nil {
return nil, err
}
// important to remove nonblock because better to handle reads and writes
if err = syscall.SetNonblock(fd, false); err != nil {
return nil, err
}
d := &Device{fd: uintptr(fd), path: path}
d.init()
// load all supported formats, channels, rates, etc.
if err = ioctl(d.fd, SNDRV_PCM_IOCTL_HW_REFINE, &d.hwparams); err != nil {
_ = d.Close()
return nil, err
}
d.setMask(SNDRV_PCM_HW_PARAM_ACCESS, SNDRV_PCM_ACCESS_RW_INTERLEAVED)
return d, nil
}
func (d *Device) Close() error {
return syscall.Close(int(d.fd))
}
func (d *Device) IsCapture() bool {
// path: /dev/snd/pcmC0D0c, where p - playback, c - capture
return d.path[len(d.path)-1] == 'c'
}
type Info struct {
Card int
Device int
SubDevice int
Stream int
ID string
Name string
SubName string
}
func (d *Device) Info() (*Info, error) {
var info snd_pcm_info
if err := ioctl(d.fd, SNDRV_PCM_IOCTL_INFO, &info); err != nil {
return nil, err
}
return &Info{
Card: int(info.card),
Device: int(info.device),
SubDevice: int(info.subdevice),
Stream: int(info.stream),
ID: str(info.id[:]),
Name: str(info.name[:]),
SubName: str(info.subname[:]),
}, nil
}
func (d *Device) CheckFormat(format byte) bool {
return d.checkMask(SNDRV_PCM_HW_PARAM_FORMAT, uint32(format))
}
func (d *Device) ListFormats() (formats []byte) {
for i := byte(0); i <= 28; i++ {
if d.CheckFormat(i) {
formats = append(formats, i)
}
}
return
}
func (d *Device) RangeRates() (uint32, uint32) {
return d.getInterval(SNDRV_PCM_HW_PARAM_RATE)
}
func (d *Device) RangeChannels() (byte, byte) {
minCh, maxCh := d.getInterval(SNDRV_PCM_HW_PARAM_CHANNELS)
return byte(minCh), byte(maxCh)
}
func (d *Device) GetRateNear(rate uint32) uint32 {
r1, r2 := d.RangeRates()
if rate < r1 {
return r1
}
if rate > r2 {
return r2
}
return rate
}
func (d *Device) GetChannelsNear(channels byte) byte {
c1, c2 := d.RangeChannels()
if channels < c1 {
return c1
}
if channels > c2 {
return c2
}
return channels
}
const bufferSize = 4096
func (d *Device) SetHWParams(format byte, rate uint32, channels byte) error {
d.setInterval(SNDRV_PCM_HW_PARAM_CHANNELS, uint32(channels))
d.setInterval(SNDRV_PCM_HW_PARAM_RATE, rate)
d.setMask(SNDRV_PCM_HW_PARAM_FORMAT, uint32(format))
//d.setMask(SNDRV_PCM_HW_PARAM_SUBFORMAT, 0)
// important for smooth playback
d.setInterval(SNDRV_PCM_HW_PARAM_BUFFER_SIZE, bufferSize)
//d.setInterval(SNDRV_PCM_HW_PARAM_PERIOD_SIZE, 2000)
if err := ioctl(d.fd, SNDRV_PCM_IOCTL_HW_PARAMS, &d.hwparams); err != nil {
return fmt.Errorf("[alsa] set hw_params: %w", err)
}
_, i := d.getInterval(SNDRV_PCM_HW_PARAM_FRAME_BITS)
d.frameBytes = int(i / 8)
_, periods := d.getInterval(SNDRV_PCM_HW_PARAM_PERIODS)
_, periodSize := d.getInterval(SNDRV_PCM_HW_PARAM_PERIOD_SIZE)
threshold := snd_pcm_uframes_t(periods * periodSize) // same as bufferSize
swparams := snd_pcm_sw_params{
//tstamp_mode: SNDRV_PCM_TSTAMP_ENABLE,
period_step: 1,
avail_min: 1, // start as soon as possible
stop_threshold: threshold,
}
if d.IsCapture() {
swparams.start_threshold = 1
} else {
swparams.start_threshold = threshold
}
if err := ioctl(d.fd, SNDRV_PCM_IOCTL_SW_PARAMS, &swparams); err != nil {
return fmt.Errorf("[alsa] set sw_params: %w", err)
}
if err := ioctl(d.fd, SNDRV_PCM_IOCTL_PREPARE, nil); err != nil {
return fmt.Errorf("[alsa] prepare: %w", err)
}
return nil
}
func (d *Device) Write(b []byte) (n int, err error) {
xfer := &snd_xferi{
buf: uintptr(unsafe.Pointer(&b[0])),
frames: snd_pcm_uframes_t(len(b) / d.frameBytes),
}
err = ioctl(d.fd, SNDRV_PCM_IOCTL_WRITEI_FRAMES, xfer)
if err == syscall.EPIPE {
// auto handle underrun state
// https://stackoverflow.com/questions/59396728/how-to-properly-handle-xrun-in-alsa-programming-when-playing-audio-with-snd-pcm
err = ioctl(d.fd, SNDRV_PCM_IOCTL_PREPARE, nil)
}
n = int(xfer.result) * d.frameBytes
return
}
func (d *Device) Read(b []byte) (n int, err error) {
xfer := &snd_xferi{
buf: uintptr(unsafe.Pointer(&b[0])),
frames: snd_pcm_uframes_t(len(b) / d.frameBytes),
}
err = ioctl(d.fd, SNDRV_PCM_IOCTL_READI_FRAMES, xfer)
n = int(xfer.result) * d.frameBytes
return
}
func (d *Device) init() {
for i := range d.hwparams.masks {
d.hwparams.masks[i].bits[0] = 0xFFFFFFFF
d.hwparams.masks[i].bits[1] = 0xFFFFFFFF
}
for i := range d.hwparams.intervals {
d.hwparams.intervals[i].max = 0xFFFFFFFF
}
d.hwparams.rmask = 0xFFFFFFFF
d.hwparams.cmask = 0
d.hwparams.info = 0xFFFFFFFF
}
func (d *Device) setInterval(param, val uint32) {
d.hwparams.intervals[param-SNDRV_PCM_HW_PARAM_FIRST_INTERVAL].min = val
d.hwparams.intervals[param-SNDRV_PCM_HW_PARAM_FIRST_INTERVAL].max = val
d.hwparams.intervals[param-SNDRV_PCM_HW_PARAM_FIRST_INTERVAL].bit = 0b0100 // integer
}
func (d *Device) setIntervalMin(param, val uint32) {
d.hwparams.intervals[param-SNDRV_PCM_HW_PARAM_FIRST_INTERVAL].min = val
}
func (d *Device) getInterval(param uint32) (uint32, uint32) {
return d.hwparams.intervals[param-SNDRV_PCM_HW_PARAM_FIRST_INTERVAL].min,
d.hwparams.intervals[param-SNDRV_PCM_HW_PARAM_FIRST_INTERVAL].max
}
func (d *Device) setMask(mask, val uint32) {
d.hwparams.masks[mask].bits[0] = 0
d.hwparams.masks[mask].bits[1] = 0
d.hwparams.masks[mask].bits[val>>5] = 1 << (val & 0x1F)
}
func (d *Device) checkMask(mask, val uint32) bool {
return d.hwparams.masks[mask].bits[val>>5]&(1<<(val&0x1F)) > 0
}
@@ -0,0 +1,26 @@
package device
import (
"bytes"
"reflect"
"syscall"
)
func ioctl(fd, req uintptr, arg any) error {
var ptr uintptr
if arg != nil {
ptr = reflect.ValueOf(arg).Pointer()
}
_, _, err := syscall.Syscall(syscall.SYS_IOCTL, fd, req, ptr)
if err != 0 {
return err
}
return nil
}
func str(b []byte) string {
if i := bytes.IndexByte(b, 0); i >= 0 {
return string(b[:i])
}
return string(b)
}
@@ -0,0 +1,44 @@
package alsa
import (
"errors"
"fmt"
"net/url"
"github.com/AlexxIT/go2rtc/pkg/alsa/device"
"github.com/AlexxIT/go2rtc/pkg/core"
)
func Open(rawURL string) (core.Producer, error) {
// Example (ffmpeg source compatible):
// alsa:device?audio=/dev/snd/pcmC0D0p
// TODO: ?audio=default
// TODO: ?audio=hw:0,0
// TODO: &sample_rate=48000&channels=2
// TODO: &backchannel=1
u, err := url.Parse(rawURL)
if err != nil {
return nil, err
}
path := u.Query().Get("audio")
dev, err := device.Open(path)
if err != nil {
return nil, err
}
if !dev.CheckFormat(device.SNDRV_PCM_FORMAT_S16_LE) {
_ = dev.Close()
return nil, errors.New("alsa: format S16LE not supported")
}
switch path[len(path)-1] {
case 'p': // playback
return newPlayback(dev)
case 'c': // capture
return newCapture(dev)
}
_ = dev.Close()
return nil, fmt.Errorf("alsa: unknown path: %s", path)
}
@@ -0,0 +1,84 @@
package alsa
import (
"fmt"
"github.com/AlexxIT/go2rtc/pkg/alsa/device"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/pcm"
"github.com/pion/rtp"
)
type Playback struct {
core.Connection
dev *device.Device
closed core.Waiter
}
func newPlayback(dev *device.Device) (*Playback, error) {
medias := []*core.Media{
{
Kind: core.KindAudio,
Direction: core.DirectionSendonly,
Codecs: []*core.Codec{
{Name: core.CodecPCML}, // support ffmpeg producer (auto transcode)
{Name: core.CodecPCMA, ClockRate: 8000}, // support webrtc producer
},
},
}
return &Playback{
Connection: core.Connection{
ID: core.NewID(),
FormatName: "alsa",
Medias: medias,
Transport: dev,
},
dev: dev,
}, nil
}
func (p *Playback) GetTrack(media *core.Media, codec *core.Codec) (*core.Receiver, error) {
return nil, core.ErrCantGetTrack
}
func (p *Playback) AddTrack(media *core.Media, codec *core.Codec, track *core.Receiver) error {
src := track.Codec
dst := &core.Codec{
Name: core.CodecPCML,
ClockRate: p.dev.GetRateNear(src.ClockRate),
Channels: p.dev.GetChannelsNear(src.Channels),
}
sender := core.NewSender(media, dst)
sender.Handler = func(pkt *rtp.Packet) {
if n, err := p.dev.Write(pkt.Payload); err == nil {
p.Send += n
}
}
if sender.Handler = pcm.TranscodeHandler(dst, src, sender.Handler); sender.Handler == nil {
return fmt.Errorf("alsa: can't convert %s to %s", src, dst)
}
// typical card support:
// - Formats: S16_LE, S32_LE
// - ClockRates: 8000 - 192000
// - Channels: 2 - 10
err := p.dev.SetHWParams(device.SNDRV_PCM_FORMAT_S16_LE, dst.ClockRate, byte(dst.Channels))
if err != nil {
return err
}
sender.HandleRTP(track)
p.Senders = append(p.Senders, sender)
return nil
}
func (p *Playback) Start() (err error) {
return p.closed.Wait()
}
func (p *Playback) Stop() error {
p.closed.Done(nil)
return p.Connection.Stop()
}
@@ -0,0 +1,6 @@
## Useful links
- https://en.wikipedia.org/wiki/ANSI_escape_code
- https://paulbourke.net/dataformats/asciiart/
- https://github.com/kutuluk/xterm-color-chart
- https://github.com/hugomd/parrot.live
+173
View File
@@ -0,0 +1,173 @@
package ascii
import (
"bytes"
"fmt"
"image/jpeg"
"io"
"net/http"
"unicode/utf8"
)
func NewWriter(w io.Writer, foreground, background, text string) io.Writer {
// once clear screen
_, _ = w.Write([]byte(csiClear))
// every frame - move to home
a := &writer{wr: w, buf: []byte(csiHome)}
// https://en.wikipedia.org/wiki/ANSI_escape_code
switch foreground {
case "":
case "8":
a.color = func(r, g, b uint8) {
idx := xterm256color(r, g, b, 8)
a.appendEsc(fmt.Sprintf("\033[%dm", 30+idx))
}
case "256":
a.color = func(r, g, b uint8) {
idx := xterm256color(r, g, b, 255)
a.appendEsc(fmt.Sprintf("\033[38;5;%dm", idx))
}
case "rgb":
a.color = func(r, g, b uint8) {
a.appendEsc(fmt.Sprintf("\033[38;2;%d;%d;%dm", r, g, b))
}
default:
a.buf = append(a.buf, "\033["+foreground+"m"...)
}
switch background {
case "":
case "8":
a.color = func(r, g, b uint8) {
idx := xterm256color(r, g, b, 8)
a.appendEsc(fmt.Sprintf("\033[%dm", 40+idx))
}
case "256":
a.color = func(r, g, b uint8) {
idx := xterm256color(r, g, b, 255)
a.appendEsc(fmt.Sprintf("\033[48;5;%dm", idx))
}
case "rgb":
a.color = func(r, g, b uint8) {
a.appendEsc(fmt.Sprintf("\033[48;2;%d;%d;%dm", r, g, b))
}
default:
a.buf = append(a.buf, "\033["+background+"m"...)
}
a.pre = len(a.buf) // save prefix size
if len(text) == 1 {
// fast 1 symbol version
a.text = func(_, _, _ uint32) {
a.buf = append(a.buf, text[0])
}
} else {
switch text {
case "":
text = ` .::--~~==++**##%%$@` // default for empty text
case "block":
text = " ░░▒▒▓▓█" // https://en.wikipedia.org/wiki/Block_Elements
}
if runes := []rune(text); len(runes) != len(text) {
k := float32(len(runes)-1) / 255
a.text = func(r, g, b uint32) {
i := gray(r, g, b, k)
a.buf = utf8.AppendRune(a.buf, runes[i])
}
} else {
k := float32(len(text)-1) / 255
a.text = func(r, g, b uint32) {
i := gray(r, g, b, k)
a.buf = append(a.buf, text[i])
}
}
}
return a
}
type writer struct {
wr io.Writer
buf []byte
pre int
esc string
color func(r, g, b uint8)
text func(r, g, b uint32)
}
// https://stackoverflow.com/questions/37774983/clearing-the-screen-by-printing-a-character
const csiClear = "\033[2J"
const csiHome = "\033[H"
func (a *writer) Write(p []byte) (n int, err error) {
img, err := jpeg.Decode(bytes.NewReader(p))
if err != nil {
return 0, err
}
a.buf = a.buf[:a.pre] // restore prefix
w := img.Bounds().Dx()
h := img.Bounds().Dy()
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
r, g, b, _ := img.At(x, y).RGBA()
if a.color != nil {
a.color(uint8(r>>8), uint8(g>>8), uint8(b>>8))
}
a.text(r, g, b)
}
a.buf = append(a.buf, '\n')
}
a.appendEsc("\033[0m")
if _, err = a.wr.Write(a.buf); err != nil {
return 0, err
}
a.wr.(http.Flusher).Flush()
return len(p), nil
}
// appendEsc - append ESC code to buffer, and skip duplicates
func (a *writer) appendEsc(s string) {
if a.esc != s {
a.esc = s
a.buf = append(a.buf, s...)
}
}
func gray(r, g, b uint32, k float32) uint8 {
gr := (19595*r + 38470*g + 7471*b + 1<<15) >> 24 // uint8
return uint8(float32(gr) * k)
}
const x256r = "\x00\x80\x00\x80\x00\x80\x00\xc0\x80\xff\x00\xff\x00\xff\x00\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x5f\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\x87\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xaf\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xd7\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x08\x12\x1c\x26\x30\x3a\x44\x4e\x58\x60\x66\x76\x80\x8a\x94\x9e\xa8\xb2\xbc\xc6\xd0\xda\xe4\xee"
const x256g = "\x00\x00\x80\x80\x00\x00\x80\xc0\x80\x00\xff\xff\x00\x00\xff\xff\x00\x00\x00\x00\x00\x00\x5f\x5f\x5f\x5f\x5f\x5f\x87\x87\x87\x87\x87\x87\xaf\xaf\xaf\xaf\xaf\xaf\xd7\xd7\xd7\xd7\xd7\xd7\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x5f\x5f\x5f\x5f\x5f\x5f\x87\x87\x87\x87\x87\x87\xaf\xaf\xaf\xaf\xaf\xaf\xd7\xd7\xd7\xd7\xd7\xd7\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x5f\x5f\x5f\x5f\x5f\x5f\x87\x87\x87\x87\x87\x87\xaf\xaf\xaf\xaf\xaf\xaf\xd7\xd7\xd7\xd7\xd7\xd7\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x5f\x5f\x5f\x5f\x5f\x5f\x87\x87\x87\x87\x87\x87\xaf\xaf\xaf\xaf\xaf\xaf\xd7\xd7\xd7\xd7\xd7\xd7\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x5f\x5f\x5f\x5f\x5f\x5f\x87\x87\x87\x87\x87\x87\xaf\xaf\xaf\xaf\xaf\xaf\xd7\xd7\xd7\xd7\xd7\xd7\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x5f\x5f\x5f\x5f\x5f\x5f\x87\x87\x87\x87\x87\x87\xaf\xaf\xaf\xaf\xaf\xaf\xd7\xd7\xd7\xd7\xd7\xd7\xff\xff\xff\xff\xff\xff\x08\x12\x1c\x26\x30\x3a\x44\x4e\x58\x60\x66\x76\x80\x8a\x94\x9e\xa8\xb2\xbc\xc6\xd0\xda\xe4\xee"
const x256b = "\x00\x00\x00\x00\x80\x80\x80\xc0\x80\x00\x00\x00\xff\xff\xff\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x00\x5f\x87\xaf\xd7\xff\x08\x12\x1c\x26\x30\x3a\x44\x4e\x58\x60\x66\x76\x80\x8a\x94\x9e\xa8\xb2\xbc\xc6\xd0\xda\xe4\xee"
func xterm256color(r, g, b uint8, n int) (index uint8) {
best := uint16(0xFFFF)
for i := 0; i < n; i++ {
diff := sqDiff(r, x256r[i]) + sqDiff(g, x256g[i]) + sqDiff(b, x256b[i])
if diff < best {
best = diff
index = uint8(i)
}
}
return
}
// sqDiff - just like from image/color/color.go
func sqDiff(x, y uint8) uint16 {
d := uint16(x - y)
//return d
return (d * d) >> 2
}
+143
View File
@@ -0,0 +1,143 @@
package bits
type Reader struct {
EOF bool // if end of buffer raised during reading
buf []byte // total buf
byte byte // current byte
bits byte // bits left in byte
pos int // current pos in buf
}
func NewReader(b []byte) *Reader {
return &Reader{buf: b}
}
//goland:noinspection GoStandardMethods
func (r *Reader) ReadByte() byte {
if r.bits != 0 {
return r.ReadBits8(8)
}
if r.pos >= len(r.buf) {
r.EOF = true
return 0
}
b := r.buf[r.pos]
r.pos++
return b
}
func (r *Reader) ReadUint16() uint16 {
if r.bits != 0 {
return r.ReadBits16(16)
}
return uint16(r.ReadByte())<<8 | uint16(r.ReadByte())
}
func (r *Reader) ReadUint24() uint32 {
if r.bits != 0 {
return r.ReadBits(24)
}
return uint32(r.ReadByte())<<16 | uint32(r.ReadByte())<<8 | uint32(r.ReadByte())
}
func (r *Reader) ReadUint32() uint32 {
if r.bits != 0 {
return r.ReadBits(32)
}
return uint32(r.ReadByte())<<24 | uint32(r.ReadByte())<<16 | uint32(r.ReadByte())<<8 | uint32(r.ReadByte())
}
func (r *Reader) ReadBit() byte {
if r.bits == 0 {
r.byte = r.ReadByte()
r.bits = 7
} else {
r.bits--
}
return (r.byte >> r.bits) & 0b1
}
func (r *Reader) ReadBits(n byte) (res uint32) {
for i := n - 1; i != 255; i-- {
res |= uint32(r.ReadBit()) << i
}
return
}
func (r *Reader) ReadBits8(n byte) (res uint8) {
for i := n - 1; i != 255; i-- {
res |= r.ReadBit() << i
}
return
}
func (r *Reader) ReadBits16(n byte) (res uint16) {
for i := n - 1; i != 255; i-- {
res |= uint16(r.ReadBit()) << i
}
return
}
func (r *Reader) ReadBits64(n byte) (res uint64) {
for i := n - 1; i != 255; i-- {
res |= uint64(r.ReadBit()) << i
}
return
}
func (r *Reader) ReadFloat32() float64 {
i := r.ReadUint16()
f := r.ReadUint16()
return float64(i) + float64(f)/65536
}
func (r *Reader) ReadBytes(n int) (b []byte) {
if r.bits == 0 {
if r.pos+n > len(r.buf) {
r.EOF = true
return nil
}
b = r.buf[r.pos : r.pos+n]
r.pos += n
} else {
b = make([]byte, n)
for i := 0; i < n; i++ {
b[i] = r.ReadByte()
}
}
return
}
// ReadUEGolomb - ReadExponentialGolomb (unsigned)
func (r *Reader) ReadUEGolomb() uint32 {
var size byte
for size = 0; size < 32; size++ {
if b := r.ReadBit(); b != 0 || r.EOF {
break
}
}
return r.ReadBits(size) + (1 << size) - 1
}
// ReadSEGolomb - ReadSignedExponentialGolomb
func (r *Reader) ReadSEGolomb() int32 {
if b := r.ReadUEGolomb(); b%2 == 0 {
return -int32(b / 2)
} else {
return int32((b + 1) / 2)
}
}
func (r *Reader) Left() []byte {
return r.buf[r.pos:]
}
func (r *Reader) Pos() (int, byte) {
return r.pos - 1, r.bits
}
@@ -0,0 +1,95 @@
package bits
type Writer struct {
buf []byte // total buf
byte *byte // pointer to current byte
bits byte // bits left in byte
}
func NewWriter(buf []byte) *Writer {
return &Writer{buf: buf}
}
//goland:noinspection GoStandardMethods
func (w *Writer) WriteByte(b byte) {
if w.bits != 0 {
w.WriteBits8(b, 8)
}
w.buf = append(w.buf, b)
}
func (w *Writer) WriteBit(b byte) {
if w.bits == 0 {
w.buf = append(w.buf, 0)
w.byte = &w.buf[len(w.buf)-1]
w.bits = 7
} else {
w.bits--
}
*w.byte |= (b & 1) << w.bits
}
func (w *Writer) WriteBits(v uint32, n byte) {
for i := n - 1; i != 255; i-- {
w.WriteBit(byte(v>>i) & 0b1)
}
}
func (w *Writer) WriteBits16(v uint16, n byte) {
for i := n - 1; i != 255; i-- {
w.WriteBit(byte(v>>i) & 0b1)
}
}
func (w *Writer) WriteBits8(v, n byte) {
for i := n - 1; i != 255; i-- {
w.WriteBit((v >> i) & 0b1)
}
}
func (w *Writer) WriteAllBits(bit, n byte) {
for i := byte(0); i < n; i++ {
w.WriteBit(bit)
}
}
func (w *Writer) WriteBool(b bool) {
if b {
w.WriteBit(1)
} else {
w.WriteBit(0)
}
}
func (w *Writer) WriteUint16(v uint16) {
if w.bits != 0 {
w.WriteBits16(v, 16)
}
w.buf = append(w.buf, byte(v>>8), byte(v))
}
func (w *Writer) WriteBytes(bytes ...byte) {
if w.bits != 0 {
for _, b := range bytes {
w.WriteByte(b)
}
}
w.buf = append(w.buf, bytes...)
}
func (w *Writer) Bytes() []byte {
return w.buf
}
func (w *Writer) Len() int {
return len(w.buf)
}
func (w *Writer) Reset() {
w.buf = w.buf[:0]
w.bits = 0
}
@@ -0,0 +1,266 @@
// Package bubble, because:
// Request URL: /bubble/live?ch=0&stream=0
// Response Conten-Type: video/bubble
// https://github.com/Lynch234ok/lynch-git/blob/master/app_rebulid/src/bubble.c
package bubble
import (
"bufio"
"encoding/binary"
"errors"
"io"
"net"
"net/http"
"net/url"
"regexp"
"strings"
"time"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/h264/annexb"
"github.com/AlexxIT/go2rtc/pkg/tcp"
"github.com/pion/rtp"
)
// Deprecated: should be rewritten to core.Connection
type Client struct {
core.Listener
url string
conn net.Conn
videoCodec string
channel int
stream int
r *bufio.Reader
medias []*core.Media
receivers []*core.Receiver
videoTrack *core.Receiver
audioTrack *core.Receiver
recv int
}
func Dial(rawURL string) (*Client, error) {
client := &Client{url: rawURL}
if err := client.Dial(); err != nil {
return nil, err
}
return client, nil
}
const (
SyncByte = 0xAA
PacketAuth = 0x00
PacketMedia = 0x01
PacketStart = 0x0A
)
const Timeout = time.Second * 5
func (c *Client) Dial() (err error) {
u, err := url.Parse(c.url)
if err != nil {
return
}
if c.conn, err = net.DialTimeout("tcp", u.Host, Timeout); err != nil {
return
}
if err = c.conn.SetDeadline(time.Now().Add(Timeout)); err != nil {
return
}
req := &tcp.Request{Method: "GET", URL: &url.URL{Path: u.Path, RawQuery: u.RawQuery}, Proto: "HTTP/1.1"}
if err = req.Write(c.conn); err != nil {
return
}
c.r = bufio.NewReader(c.conn)
res, err := tcp.ReadResponse(c.r)
if err != nil {
return
}
if res.StatusCode != http.StatusOK {
return errors.New("wrong response: " + res.Status)
}
// 1. Read 1024 bytes with XML, some cameras returns exact 1024, but some - 923
xml := make([]byte, 1024)
if _, err = c.r.Read(xml); err != nil {
return
}
// 2. Write size uint32 + unknown 4b + user 20b + pass 20b
b := make([]byte, 48)
binary.BigEndian.PutUint32(b, 44)
if u.User != nil {
copy(b[8:], u.User.Username())
pass, _ := u.User.Password()
copy(b[28:], pass)
} else {
copy(b[8:], "admin")
}
if err = c.Write(PacketAuth, 0x0E16C271, b); err != nil {
return
}
// 3. Read response
cmd, b, err := c.Read()
if err != nil {
return
}
if cmd != PacketAuth || len(b) != 44 || b[4] != 3 || b[8] != 1 {
return errors.New("wrong auth response")
}
// 4. Parse XML (from 1)
query := u.Query()
stream := query.Get("stream")
if stream != "" {
c.stream = core.Atoi(stream)
} else {
stream = "0"
}
// <bubble version="1.0" vin="1"><vin0 stream="2">
// <stream0 name="720p.264" size="2304x1296" x1="yes" x2="yes" x4="yes" />
// <stream1 name="360p.265" size="640x360" x1="yes" x2="yes" x4="yes" />
// <vin0>
// </bubble>
re := regexp.MustCompile("<stream" + stream + " [^>]+")
stream = re.FindString(string(xml))
if strings.Contains(stream, ".265") {
c.videoCodec = core.CodecH265
} else {
c.videoCodec = core.CodecH264
}
if ch := query.Get("ch"); ch != "" {
c.channel = core.Atoi(ch)
}
return
}
func (c *Client) Write(command byte, timestamp uint32, payload []byte) error {
if err := c.conn.SetWriteDeadline(time.Now().Add(Timeout)); err != nil {
return err
}
// 0xAA + size uint32 + cmd byte + ts uint32 + payload
b := make([]byte, 14+len(payload))
b[0] = SyncByte
binary.BigEndian.PutUint32(b[1:], uint32(5+len(payload)))
b[5] = command
binary.BigEndian.PutUint32(b[6:], timestamp)
copy(b[10:], payload)
_, err := c.conn.Write(b)
return err
}
func (c *Client) Read() (byte, []byte, error) {
if err := c.conn.SetReadDeadline(time.Now().Add(Timeout)); err != nil {
return 0, nil, err
}
// 0xAA + size uint32 + cmd byte + ts uint32 + payload
b := make([]byte, 10)
if _, err := io.ReadFull(c.r, b); err != nil {
return 0, nil, err
}
if b[0] != SyncByte {
return 0, nil, errors.New("wrong start byte")
}
size := binary.BigEndian.Uint32(b[1:])
payload := make([]byte, size-1-4)
if _, err := io.ReadFull(c.r, payload); err != nil {
return 0, nil, err
}
//timestamp := binary.BigEndian.Uint32(b[6:]) // in ms
return b[5], payload, nil
}
func (c *Client) Play() error {
// yeah, there's no mistake about the little endian
b := make([]byte, 16)
binary.LittleEndian.PutUint32(b, uint32(c.channel))
binary.LittleEndian.PutUint32(b[4:], uint32(c.stream))
binary.LittleEndian.PutUint32(b[8:], 1) // opened
return c.Write(PacketStart, 0x0E16C2DF, b)
}
func (c *Client) Handle() error {
var audioTS uint32
for {
cmd, b, err := c.Read()
if err != nil {
return err
}
c.recv += len(b)
if cmd != PacketMedia {
continue
}
// size uint32 + type 1b + channel 1b
// type = 1 for keyframe, 2 for other frame, 0 for audio
if b[4] > 0 {
if c.videoTrack == nil {
continue
}
pkt := &rtp.Packet{
Header: rtp.Header{
Timestamp: core.Now90000(),
},
Payload: annexb.EncodeToAVCC(b[6:]),
}
c.videoTrack.WriteRTP(pkt)
} else {
if c.audioTrack == nil {
continue
}
//binary.LittleEndian.Uint32(b[6:]) // entries (always 1)
//size := binary.LittleEndian.Uint32(b[10:]) // size
//mk := binary.LittleEndian.Uint64(b[14:]) // pts (uint64_t)
//binary.LittleEndian.Uint32(b[22:]) // gtime (time_t)
//name := b[26:34] // g711
//rate := binary.LittleEndian.Uint32(b[34:]) // sample rate
//width := binary.LittleEndian.Uint32(b[38:]) // samplewidth
pkt := &rtp.Packet{
Header: rtp.Header{
Version: 2,
Marker: true,
Timestamp: audioTS,
},
Payload: b[6+36:],
}
audioTS += uint32(len(pkt.Payload))
c.audioTrack.WriteRTP(pkt)
}
}
}
func (c *Client) Close() error {
return c.conn.Close()
}
@@ -0,0 +1,80 @@
package bubble
import (
"encoding/json"
"github.com/AlexxIT/go2rtc/pkg/core"
)
func (c *Client) GetMedias() []*core.Media {
if c.medias == nil {
c.medias = []*core.Media{
{
Kind: core.KindVideo,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{
{Name: c.videoCodec, ClockRate: 90000, PayloadType: core.PayloadTypeRAW},
},
},
{
Kind: core.KindAudio,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{
{Name: core.CodecPCMA, ClockRate: 8000, PayloadType: 8},
},
},
}
}
return c.medias
}
func (c *Client) GetTrack(media *core.Media, codec *core.Codec) (*core.Receiver, error) {
for _, track := range c.receivers {
if track.Codec == codec {
return track, nil
}
}
track := core.NewReceiver(media, codec)
switch media.Kind {
case core.KindVideo:
c.videoTrack = track
case core.KindAudio:
c.audioTrack = track
}
c.receivers = append(c.receivers, track)
return track, nil
}
func (c *Client) Start() error {
if err := c.Play(); err != nil {
return err
}
return c.Handle()
}
func (c *Client) Stop() error {
for _, receiver := range c.receivers {
receiver.Close()
}
return c.Close()
}
func (c *Client) MarshalJSON() ([]byte, error) {
info := &core.Connection{
ID: core.ID(c),
FormatName: "bubble",
Protocol: "http",
Medias: c.medias,
Recv: c.recv,
Receivers: c.receivers,
}
if c.conn != nil {
info.RemoteAddr = c.conn.RemoteAddr().String()
}
return json.Marshal(info)
}
@@ -0,0 +1,40 @@
## PCM
**RTSP**
- PayloadType=10 - L16/44100/2 - Linear PCM 16-bit big endian
- PayloadType=11 - L16/44100/1 - Linear PCM 16-bit big endian
https://en.wikipedia.org/wiki/RTP_payload_formats
**Apple QuickTime**
- `raw` - 16-bit data is stored in little endian format
- `twos` - 16-bit data is stored in big endian format
- `sowt` - 16-bit data is stored in little endian format
- `in24` - denotes 24-bit, big endian
- `in32` - denotes 32-bit, big endian
- `fl32` - denotes 32-bit floating point PCM
- `fl64` - denotes 64-bit floating point PCM
- `alaw` - denotes A-law logarithmic PCM
- `ulaw` - denotes mu-law logarithmic PCM
https://wiki.multimedia.cx/index.php/PCM
**FFmpeg RTSP**
```
pcm_s16be, 44100 Hz, stereo => 10
pcm_s16be, 48000 Hz, stereo => 96 L16/48000/2
pcm_s16be, 44100 Hz, mono => 11
pcm_s16le, 48000 Hz, stereo => 96 (b=AS:1536)
pcm_s16le, 44100 Hz, stereo => 96 (b=AS:1411)
pcm_s16le, 16000 Hz, stereo => 96 (b=AS:512)
pcm_s16le, 8000 Hz, stereo => 96 (b=AS:256)
pcm_s16le, 48000 Hz, mono => 96 (b=AS:768)
pcm_s16le, 44100 Hz, mono => 96 (b=AS:705)
pcm_s16le, 16000 Hz, mono => 96 (b=AS:256)
pcm_s16le, 8000 Hz, mono => 96 (b=AS:128)
```
+284
View File
@@ -0,0 +1,284 @@
package core
import (
"encoding/base64"
"encoding/json"
"fmt"
"strings"
"unicode"
"github.com/pion/sdp/v3"
)
type Codec struct {
Name string // H264, PCMU, PCMA, opus...
ClockRate uint32 // 90000, 8000, 16000...
Channels uint8 // 0, 1, 2
FmtpLine string
PayloadType uint8
}
// MarshalJSON - return FFprobe compatible output
func (c *Codec) MarshalJSON() ([]byte, error) {
info := map[string]any{}
if name := FFmpegCodecName(c.Name); name != "" {
info["codec_name"] = name
info["codec_type"] = c.Kind()
}
if c.Name == CodecH264 {
profile, level := DecodeH264(c.FmtpLine)
if profile != "" {
info["profile"] = profile
info["level"] = level
}
}
if c.ClockRate != 0 && c.ClockRate != 90000 {
info["sample_rate"] = c.ClockRate
}
if c.Channels > 0 {
info["channels"] = c.Channels
}
return json.Marshal(info)
}
func FFmpegCodecName(name string) string {
switch name {
case CodecH264:
return "h264"
case CodecH265:
return "hevc"
case CodecJPEG:
return "mjpeg"
case CodecRAW:
return "rawvideo"
case CodecPCMA:
return "pcm_alaw"
case CodecPCMU:
return "pcm_mulaw"
case CodecPCM:
return "pcm_s16be"
case CodecPCML:
return "pcm_s16le"
case CodecAAC:
return "aac"
case CodecOpus:
return "opus"
case CodecVP8:
return "vp8"
case CodecVP9:
return "vp9"
case CodecAV1:
return "av1"
case CodecELD:
return "aac/eld"
case CodecFLAC:
return "flac"
case CodecMP3:
return "mp3"
}
return name
}
func (c *Codec) String() (s string) {
s = c.Name
if c.ClockRate != 0 && c.ClockRate != 90000 {
s += fmt.Sprintf("/%d", c.ClockRate)
}
if c.Channels > 0 {
s += fmt.Sprintf("/%d", c.Channels)
}
return
}
func (c *Codec) IsRTP() bool {
return c.PayloadType != PayloadTypeRAW
}
func (c *Codec) IsVideo() bool {
return c.Kind() == KindVideo
}
func (c *Codec) IsAudio() bool {
return c.Kind() == KindAudio
}
func (c *Codec) Kind() string {
return GetKind(c.Name)
}
func (c *Codec) PrintName() string {
switch c.Name {
case CodecAAC:
return "AAC"
case CodecPCM:
return "S16B"
case CodecPCML:
return "S16L"
}
return c.Name
}
func (c *Codec) Clone() *Codec {
clone := *c
return &clone
}
func (c *Codec) Match(remote *Codec) bool {
switch remote.Name {
case CodecAll, CodecAny:
return true
}
return c.Name == remote.Name &&
(c.ClockRate == remote.ClockRate || remote.ClockRate == 0) &&
(c.Channels == remote.Channels || remote.Channels == 0)
}
func UnmarshalCodec(md *sdp.MediaDescription, payloadType string) *Codec {
c := &Codec{PayloadType: byte(Atoi(payloadType))}
for _, attr := range md.Attributes {
switch {
case c.Name == "" && attr.Key == "rtpmap" && strings.HasPrefix(attr.Value, payloadType):
i := strings.IndexByte(attr.Value, ' ')
ss := strings.Split(attr.Value[i+1:], "/")
c.Name = strings.ToUpper(ss[0])
// fix tailing space: `a=rtpmap:96 H264/90000 `
c.ClockRate = uint32(Atoi(strings.TrimRightFunc(ss[1], unicode.IsSpace)))
if len(ss) == 3 && ss[2] == "2" {
c.Channels = 2
}
case c.FmtpLine == "" && attr.Key == "fmtp" && strings.HasPrefix(attr.Value, payloadType):
if i := strings.IndexByte(attr.Value, ' '); i > 0 {
c.FmtpLine = attr.Value[i+1:]
}
}
}
switch c.Name {
case "PCM":
// https://www.reddit.com/r/Hikvision/comments/17elxex/comment/k642g2r/
// check pkg/rtsp/rtsp_test.go TestHikvisionPCM
c.Name = CodecPCML
case "":
// https://en.wikipedia.org/wiki/RTP_payload_formats
switch payloadType {
case "0":
c.Name = CodecPCMU
c.ClockRate = 8000
case "8":
c.Name = CodecPCMA
c.ClockRate = 8000
case "10":
c.Name = CodecPCM
c.ClockRate = 44100
c.Channels = 2
case "11":
c.Name = CodecPCM
c.ClockRate = 44100
case "14":
c.Name = CodecMP3
c.ClockRate = 90000 // it's not real sample rate
case "26":
c.Name = CodecJPEG
c.ClockRate = 90000
case "96", "97", "98":
if len(md.Bandwidth) == 0 {
c.Name = payloadType
break
}
// FFmpeg + RTSP + pcm_s16le = doesn't pass info about codec name and params
// so try to guess the codec based on bitrate
// https://github.com/AlexxIT/go2rtc/issues/523
switch md.Bandwidth[0].Bandwidth {
case 128:
c.ClockRate = 8000
case 256:
c.ClockRate = 16000
case 384:
c.ClockRate = 24000
case 512:
c.ClockRate = 32000
case 705:
c.ClockRate = 44100
case 768:
c.ClockRate = 48000
case 1411:
// default Windows DShow
c.ClockRate = 44100
c.Channels = 2
case 1536:
// default Linux ALSA
c.ClockRate = 48000
c.Channels = 2
default:
c.Name = payloadType
break
}
c.Name = CodecPCML
default:
c.Name = payloadType
}
}
return c
}
func DecodeH264(fmtp string) (profile string, level byte) {
if ps := Between(fmtp, "sprop-parameter-sets=", ","); ps != "" {
if sps, _ := base64.StdEncoding.DecodeString(ps); len(sps) >= 4 {
switch sps[1] {
case 0x42:
profile = "Baseline"
case 0x4D:
profile = "Main"
case 0x58:
profile = "Extended"
case 0x64:
profile = "High"
default:
profile = fmt.Sprintf("0x%02X", sps[1])
}
level = sps[3]
}
}
return
}
func ParseCodecString(s string) *Codec {
var codec Codec
ss := strings.Split(s, "/")
switch strings.ToLower(ss[0]) {
case "pcm_s16be", "s16be", "pcm":
codec.Name = CodecPCM
case "pcm_s16le", "s16le", "pcml":
codec.Name = CodecPCML
case "pcm_alaw", "alaw", "pcma", "g711a":
codec.Name = CodecPCMA
case "pcm_mulaw", "mulaw", "pcmu", "g711u":
codec.Name = CodecPCMU
case "aac", "mpeg4-generic":
codec.Name = CodecAAC
case "opus":
codec.Name = CodecOpus
case "flac":
codec.Name = CodecFLAC
default:
return nil
}
if len(ss) >= 2 {
codec.ClockRate = uint32(Atoi(ss[1]))
}
if len(ss) >= 3 {
codec.Channels = uint8(Atoi(ss[2]))
}
return &codec
}
@@ -0,0 +1,144 @@
package core
import (
"io"
"net/http"
"reflect"
"sync/atomic"
)
func NewID() uint32 {
return id.Add(1)
}
// Deprecated: use NewID instead
func ID(v any) uint32 {
p := uintptr(reflect.ValueOf(v).UnsafePointer())
return 0x8000_0000 | uint32(p)
}
var id atomic.Uint32
type Info interface {
SetProtocol(string)
SetRemoteAddr(string)
SetSource(string)
SetURL(string)
WithRequest(*http.Request)
GetSource() string
}
// Connection just like webrtc.PeerConnection
// - ID and RemoteAddr used for building Connection(s) graph
// - FormatName, Protocol, RemoteAddr, Source, URL, SDP, UserAgent used for info about Connection
// - FormatName and Protocol has FFmpeg compatible names
// - Transport used for auto closing on Stop
type Connection struct {
ID uint32 `json:"id,omitempty"`
FormatName string `json:"format_name,omitempty"` // rtsp, webrtc, mp4, mjpeg, mpjpeg...
Protocol string `json:"protocol,omitempty"` // tcp, udp, http, ws, pipe...
RemoteAddr string `json:"remote_addr,omitempty"` // host:port other info
Source string `json:"source,omitempty"`
URL string `json:"url,omitempty"`
SDP string `json:"sdp,omitempty"`
UserAgent string `json:"user_agent,omitempty"`
Medias []*Media `json:"medias,omitempty"`
Receivers []*Receiver `json:"receivers,omitempty"`
Senders []*Sender `json:"senders,omitempty"`
Recv int `json:"bytes_recv,omitempty"`
Send int `json:"bytes_send,omitempty"`
Transport any `json:"-"`
}
func (c *Connection) GetMedias() []*Media {
return c.Medias
}
func (c *Connection) GetTrack(media *Media, codec *Codec) (*Receiver, error) {
for _, receiver := range c.Receivers {
if receiver.Codec == codec {
return receiver, nil
}
}
receiver := NewReceiver(media, codec)
c.Receivers = append(c.Receivers, receiver)
return receiver, nil
}
func (c *Connection) Stop() error {
for _, receiver := range c.Receivers {
receiver.Close()
}
for _, sender := range c.Senders {
sender.Close()
}
if closer, ok := c.Transport.(io.Closer); ok {
return closer.Close()
}
return nil
}
// Deprecated:
func (c *Connection) Codecs() []*Codec {
codecs := make([]*Codec, len(c.Senders))
for i, sender := range c.Senders {
codecs[i] = sender.Codec
}
return codecs
}
func (c *Connection) SetProtocol(s string) {
c.Protocol = s
}
func (c *Connection) SetRemoteAddr(s string) {
if c.RemoteAddr == "" {
c.RemoteAddr = s
} else {
c.RemoteAddr += " forwarded " + s
}
}
func (c *Connection) SetSource(s string) {
c.Source = s
}
func (c *Connection) SetURL(s string) {
c.URL = s
}
func (c *Connection) WithRequest(r *http.Request) {
if r.Header.Get("Upgrade") == "websocket" {
c.Protocol = "ws"
} else {
c.Protocol = "http"
}
c.RemoteAddr = r.RemoteAddr
if remote := r.Header.Get("X-Forwarded-For"); remote != "" {
c.RemoteAddr += " forwarded " + remote
}
c.UserAgent = r.UserAgent()
}
func (c *Connection) GetSource() string {
return c.Source
}
// Create like os.Create, init Consumer with existing Transport
func Create(w io.Writer) (*Connection, error) {
return &Connection{Transport: w}, nil
}
// Open like os.Open, init Producer from existing Transport
func Open(r io.Reader) (*Connection, error) {
return &Connection{Transport: r}, nil
}
// Dial like net.Dial, init Producer via Dialing
func Dial(rawURL string) (*Connection, error) {
return &Connection{}, nil
}
+97
View File
@@ -0,0 +1,97 @@
package core
import "encoding/json"
const (
DirectionRecvonly = "recvonly"
DirectionSendonly = "sendonly"
DirectionSendRecv = "sendrecv"
)
const (
KindVideo = "video"
KindAudio = "audio"
)
const (
CodecH264 = "H264" // payloadType: 96
CodecH265 = "H265"
CodecVP8 = "VP8"
CodecVP9 = "VP9"
CodecAV1 = "AV1"
CodecJPEG = "JPEG" // payloadType: 26
CodecRAW = "RAW"
CodecPCMU = "PCMU" // payloadType: 0
CodecPCMA = "PCMA" // payloadType: 8
CodecAAC = "MPEG4-GENERIC"
CodecOpus = "OPUS" // payloadType: 111
CodecG722 = "G722"
CodecMP3 = "MPA" // payload: 14, aka MPEG-1 Layer III
CodecPCM = "L16" // Linear PCM (big endian)
CodecPCML = "PCML" // Linear PCM (little endian)
CodecELD = "ELD" // AAC-ELD
CodecFLAC = "FLAC"
CodecAll = "ALL"
CodecAny = "ANY"
)
const PayloadTypeRAW byte = 255
type Producer interface {
// GetMedias - return Media(s) with local Media.Direction:
// - recvonly for Producer Video/Audio
// - sendonly for Producer backchannel
GetMedias() []*Media
// GetTrack - return Receiver, that can only produce rtp.Packet(s)
GetTrack(media *Media, codec *Codec) (*Receiver, error)
// Deprecated: rename to Run()
Start() error
// Deprecated: rename to Close()
Stop() error
}
type Consumer interface {
// GetMedias - return Media(s) with local Media.Direction:
// - sendonly for Consumer Video/Audio
// - recvonly for Consumer backchannel
GetMedias() []*Media
AddTrack(media *Media, codec *Codec, track *Receiver) error
// Deprecated: rename to Close()
Stop() error
}
type Mode byte
const (
ModeActiveProducer Mode = iota + 1 // typical source (client)
ModePassiveConsumer
ModePassiveProducer
ModeActiveConsumer
)
func (m Mode) String() string {
switch m {
case ModeActiveProducer:
return "active producer"
case ModePassiveConsumer:
return "passive consumer"
case ModePassiveProducer:
return "passive producer"
case ModeActiveConsumer:
return "active consumer"
}
return "unknown"
}
func (m Mode) MarshalJSON() ([]byte, error) {
return json.Marshal(m.String())
}
@@ -0,0 +1,134 @@
package core
import (
"fmt"
"testing"
"github.com/stretchr/testify/require"
)
type producer struct {
Medias []*Media
Receivers []*Receiver
id byte
}
func (p *producer) GetMedias() []*Media {
return p.Medias
}
func (p *producer) GetTrack(_ *Media, codec *Codec) (*Receiver, error) {
for _, receiver := range p.Receivers {
if receiver.Codec == codec {
return receiver, nil
}
}
receiver := NewReceiver(nil, codec)
p.Receivers = append(p.Receivers, receiver)
return receiver, nil
}
func (p *producer) Start() error {
pkt := &Packet{Payload: []byte{p.id}}
p.Receivers[0].Input(pkt)
return nil
}
func (p *producer) Stop() error {
for _, receiver := range p.Receivers {
receiver.Close()
}
return nil
}
type consumer struct {
Medias []*Media
Senders []*Sender
cache chan byte
}
func (c *consumer) GetMedias() []*Media {
return c.Medias
}
func (c *consumer) AddTrack(_ *Media, _ *Codec, track *Receiver) error {
c.cache = make(chan byte, 1)
sender := NewSender(nil, track.Codec)
sender.Output = func(packet *Packet) {
c.cache <- packet.Payload[0]
}
sender.HandleRTP(track)
c.Senders = append(c.Senders, sender)
return nil
}
func (c *consumer) Stop() error {
for _, sender := range c.Senders {
sender.Close()
}
return nil
}
func (c *consumer) read() byte {
return <-c.cache
}
func TestName(t *testing.T) {
GetProducer := func(b byte) Producer {
return &producer{
Medias: []*Media{
{
Kind: KindVideo,
Direction: DirectionRecvonly,
Codecs: []*Codec{
{Name: CodecH264},
},
},
},
id: b,
}
}
// stage1
prod1 := GetProducer(1)
cons2 := &consumer{}
media1 := prod1.GetMedias()[0]
track1, _ := prod1.GetTrack(media1, media1.Codecs[0])
_ = cons2.AddTrack(nil, nil, track1)
_ = prod1.Start()
require.Equal(t, byte(1), cons2.read())
// stage2
prod2 := GetProducer(2)
media2 := prod2.GetMedias()[0]
require.NotEqual(t, fmt.Sprintf("%p", media1), fmt.Sprintf("%p", media2))
track2, _ := prod2.GetTrack(media2, media2.Codecs[0])
track1.Replace(track2)
_ = prod1.Stop()
_ = prod2.Start()
require.Equal(t, byte(2), cons2.read())
// stage3
_ = prod2.Stop()
}
func TestStripUserinfo(t *testing.T) {
s := `streams:
test:
- ffmpeg:rtsp://username:password@10.1.2.3:554/stream1
- ffmpeg:rtsp://10.1.2.3:554/stream1@#video=copy
`
s = StripUserinfo(s)
require.Equal(t, `streams:
test:
- ffmpeg:rtsp://***@10.1.2.3:554/stream1
- ffmpeg:rtsp://10.1.2.3:554/stream1@#video=copy
`, s)
}
@@ -0,0 +1,94 @@
package core
import (
"crypto/rand"
"runtime"
"strconv"
"strings"
"time"
)
const (
BufferSize = 64 * 1024 // 64K
ConnDialTimeout = 5 * time.Second
ConnDeadline = 5 * time.Second
ProbeTimeout = 5 * time.Second
)
// Now90000 - timestamp for Video (clock rate = 90000 samples per second)
func Now90000() uint32 {
return uint32(time.Duration(time.Now().UnixNano()) * 90000 / time.Second)
}
const symbols = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ-_"
// RandString base10 - numbers, base16 - hex, base36 - digits+letters
// base64 - URL safe symbols, base0 - crypto random
func RandString(size, base byte) string {
b := make([]byte, size)
if _, err := rand.Read(b); err != nil {
panic(err)
}
if base == 0 {
return string(b)
}
for i := byte(0); i < size; i++ {
b[i] = symbols[b[i]%base]
}
return string(b)
}
func Before(s, sep string) string {
if i := strings.Index(s, sep); i > 0 {
return s[:i]
}
return s
}
func Between(s, sub1, sub2 string) string {
i := strings.Index(s, sub1)
if i < 0 {
return ""
}
s = s[i+len(sub1):]
if i = strings.Index(s, sub2); i >= 0 {
return s[:i]
}
return s
}
func Atoi(s string) (i int) {
if s != "" {
i, _ = strconv.Atoi(s)
}
return
}
// ParseByte - fast parsing string to byte function
func ParseByte(s string) (b byte) {
for i, ch := range []byte(s) {
ch -= '0'
if ch > 9 {
return 0
}
if i > 0 {
b *= 10
}
b += ch
}
return
}
func Assert(ok bool) {
if !ok {
_, file, line, _ := runtime.Caller(1)
panic(file + ":" + strconv.Itoa(line))
}
}
func Caller() string {
_, file, line, _ := runtime.Caller(1)
return file + ":" + strconv.Itoa(line)
}
@@ -0,0 +1,18 @@
package core
type EventFunc func(msg any)
// Listener base struct for all classes with support feedback
type Listener struct {
events []EventFunc
}
func (l *Listener) Listen(f EventFunc) {
l.events = append(l.events, f)
}
func (l *Listener) Fire(msg any) {
for _, f := range l.events {
f(msg)
}
}
+211
View File
@@ -0,0 +1,211 @@
package core
import (
"encoding/json"
"fmt"
"strings"
"github.com/pion/sdp/v3"
)
// Media take best from:
// - deepch/vdk/format/rtsp/sdp.Media
// - pion/sdp.MediaDescription
type Media struct {
Kind string `json:"kind,omitempty"` // video or audio
Direction string `json:"direction,omitempty"` // sendonly, recvonly
Codecs []*Codec `json:"codecs,omitempty"`
ID string `json:"id,omitempty"` // MID for WebRTC, Control for RTSP
}
func (m *Media) String() string {
s := fmt.Sprintf("%s, %s", m.Kind, m.Direction)
for _, codec := range m.Codecs {
name := codec.String()
if strings.Contains(s, name) {
continue
}
s += ", " + name
}
return s
}
func (m *Media) MarshalJSON() ([]byte, error) {
return json.Marshal(m.String())
}
func (m *Media) Clone() *Media {
clone := *m
clone.Codecs = make([]*Codec, len(m.Codecs))
for i, codec := range m.Codecs {
clone.Codecs[i] = codec.Clone()
}
return &clone
}
func (m *Media) MatchMedia(remote *Media) (codec, remoteCodec *Codec) {
// check same kind and opposite dirrection
if m.Kind != remote.Kind ||
m.Direction == DirectionSendonly && remote.Direction != DirectionRecvonly ||
m.Direction == DirectionRecvonly && remote.Direction != DirectionSendonly {
return nil, nil
}
for _, codec = range m.Codecs {
for _, remoteCodec = range remote.Codecs {
if codec.Match(remoteCodec) {
return
}
}
}
return nil, nil
}
func (m *Media) MatchCodec(remote *Codec) *Codec {
for _, codec := range m.Codecs {
if codec.Match(remote) {
return codec
}
}
return nil
}
func (m *Media) MatchAll() bool {
for _, codec := range m.Codecs {
if codec.Name == CodecAll {
return true
}
}
return false
}
func (m *Media) Equal(media *Media) bool {
if media.ID != "" {
return m.ID == media.ID
}
return m.String() == media.String()
}
func GetKind(name string) string {
switch name {
case CodecH264, CodecH265, CodecVP8, CodecVP9, CodecAV1, CodecJPEG, CodecRAW:
return KindVideo
case CodecPCMU, CodecPCMA, CodecAAC, CodecOpus, CodecG722, CodecMP3, CodecPCM, CodecPCML, CodecELD, CodecFLAC:
return KindAudio
}
return ""
}
func MarshalSDP(name string, medias []*Media) ([]byte, error) {
sd := &sdp.SessionDescription{
Origin: sdp.Origin{
Username: "-", SessionID: 1, SessionVersion: 1,
NetworkType: "IN", AddressType: "IP4", UnicastAddress: "0.0.0.0",
},
SessionName: sdp.SessionName(name),
ConnectionInformation: &sdp.ConnectionInformation{
NetworkType: "IN", AddressType: "IP4", Address: &sdp.Address{
Address: "0.0.0.0",
},
},
TimeDescriptions: []sdp.TimeDescription{
{Timing: sdp.Timing{}},
},
}
for _, media := range medias {
if media.Codecs == nil {
continue
}
codec := media.Codecs[0]
switch codec.Name {
case CodecELD:
name = CodecAAC
case CodecPCML:
name = CodecPCM // beacuse we using pcm.LittleToBig for RTSP server
default:
name = codec.Name
}
md := &sdp.MediaDescription{
MediaName: sdp.MediaName{
Media: media.Kind,
Protos: []string{"RTP", "AVP"},
},
}
md.WithCodec(codec.PayloadType, name, codec.ClockRate, uint16(codec.Channels), codec.FmtpLine)
if media.Direction != "" {
md.WithPropertyAttribute(media.Direction)
}
if media.ID != "" {
md.WithValueAttribute("control", media.ID)
}
sd.MediaDescriptions = append(sd.MediaDescriptions, md)
}
return sd.Marshal()
}
func UnmarshalMedia(md *sdp.MediaDescription) *Media {
m := &Media{
Kind: md.MediaName.Media,
}
for _, attr := range md.Attributes {
switch attr.Key {
case DirectionSendonly, DirectionRecvonly, DirectionSendRecv:
m.Direction = attr.Key
case "control", "mid":
m.ID = attr.Value
}
}
for _, format := range md.MediaName.Formats {
m.Codecs = append(m.Codecs, UnmarshalCodec(md, format))
}
return m
}
func ParseQuery(query map[string][]string) (medias []*Media) {
// set media candidates from query list
for key, values := range query {
switch key {
case KindVideo, KindAudio:
for _, value := range values {
media := &Media{Kind: key, Direction: DirectionSendonly}
for _, name := range strings.Split(value, ",") {
name = strings.ToUpper(name)
// check aliases
switch name {
case "", "COPY":
name = CodecAny
case "MJPEG":
name = CodecJPEG
case "AAC":
name = CodecAAC
case "MP3":
name = CodecMP3
}
media.Codecs = append(media.Codecs, &Codec{Name: name})
}
medias = append(medias, media)
}
}
}
return
}
@@ -0,0 +1,64 @@
package core
import (
"fmt"
"net/url"
"testing"
"github.com/pion/sdp/v3"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestSDP(t *testing.T) {
medias := []*Media{{
Kind: KindAudio, Direction: DirectionSendonly,
Codecs: []*Codec{
{Name: CodecPCMU, ClockRate: 8000},
},
}}
data, err := MarshalSDP("go2rtc/1.0.0", medias)
assert.Empty(t, err)
sd := &sdp.SessionDescription{}
err = sd.Unmarshal(data)
assert.Empty(t, err)
}
func TestParseQuery(t *testing.T) {
u, _ := url.Parse("rtsp://localhost:8554/camera1")
medias := ParseQuery(u.Query())
assert.Nil(t, medias)
for _, rawULR := range []string{
"rtsp://localhost:8554/camera1?video",
"rtsp://localhost:8554/camera1?video=copy",
"rtsp://localhost:8554/camera1?video=any",
} {
u, _ = url.Parse(rawULR)
medias = ParseQuery(u.Query())
assert.Equal(t, []*Media{
{Kind: KindVideo, Direction: DirectionSendonly, Codecs: []*Codec{{Name: CodecAny}}},
}, medias)
}
}
func TestClone(t *testing.T) {
media1 := &Media{
Kind: KindVideo,
Direction: DirectionRecvonly,
Codecs: []*Codec{
{Name: CodecPCMU, ClockRate: 8000},
},
}
media2 := media1.Clone()
p1 := fmt.Sprintf("%p", media1)
p2 := fmt.Sprintf("%p", media2)
require.NotEqualValues(t, p1, p2)
p3 := fmt.Sprintf("%p", media1.Codecs[0])
p4 := fmt.Sprintf("%p", media2.Codecs[0])
require.NotEqualValues(t, p3, p4)
}
+88
View File
@@ -0,0 +1,88 @@
package core
import (
"sync"
"github.com/pion/rtp"
)
//type Packet struct {
// Payload []byte
// Timestamp uint32 // PTS if DTS == 0 else DTS
// Composition uint32 // CTS = PTS-DTS (for support B-frames)
// Sequence uint16
//}
type Packet = rtp.Packet
// HandlerFunc - process input packets (just like http.HandlerFunc)
type HandlerFunc func(packet *Packet)
// Filter - a decorator for any HandlerFunc
type Filter func(handler HandlerFunc) HandlerFunc
// Node - Receiver or Sender or Filter (transform)
type Node struct {
Codec *Codec
Input HandlerFunc
Output HandlerFunc
id uint32
childs []*Node
parent *Node
mu sync.Mutex
}
func (n *Node) WithParent(parent *Node) *Node {
parent.AppendChild(n)
return n
}
func (n *Node) AppendChild(child *Node) {
n.mu.Lock()
n.childs = append(n.childs, child)
n.mu.Unlock()
child.parent = n
}
func (n *Node) RemoveChild(child *Node) {
n.mu.Lock()
for i, ch := range n.childs {
if ch == child {
n.childs = append(n.childs[:i], n.childs[i+1:]...)
break
}
}
n.mu.Unlock()
}
func (n *Node) Close() {
if parent := n.parent; parent != nil {
parent.RemoveChild(n)
if len(parent.childs) == 0 {
parent.Close()
}
} else {
for _, childs := range n.childs {
childs.Close()
}
}
}
func MoveNode(dst, src *Node) {
src.mu.Lock()
childs := src.childs
src.childs = nil
src.mu.Unlock()
dst.mu.Lock()
dst.childs = childs
dst.mu.Unlock()
for _, child := range childs {
child.parent = dst
}
}
@@ -0,0 +1,114 @@
package core
import (
"errors"
"io"
)
// ProbeSize
// in my tests MPEG-TS 40Mbit/s 4K-video require more than 1MB for probe
const ProbeSize = 5 * 1024 * 1024 // 5MB
const (
BufferDisable = 0
BufferDrainAndClear = -1
)
// ReadBuffer support buffering and Seek over buffer
// positive BufferSize will enable buffering mode
// Seek to negative offset will clear buffer
// Seek with a positive BufferSize will continue buffering after the last read from the buffer
// Seek with a negative BufferSize will clear buffer after the last read from the buffer
// Read more than BufferSize will raise error
type ReadBuffer struct {
io.Reader
BufferSize int
buf []byte
pos int
}
func NewReadBuffer(rd io.Reader) *ReadBuffer {
if rs, ok := rd.(*ReadBuffer); ok {
return rs
}
return &ReadBuffer{Reader: rd}
}
func (r *ReadBuffer) Read(p []byte) (n int, err error) {
// with zero buffer - read as usual
if r.BufferSize == BufferDisable {
return r.Reader.Read(p)
}
// if buffer not empty - read from it
if r.pos < len(r.buf) {
n = copy(p, r.buf[r.pos:])
r.pos += n
return
}
// with negative buffer - empty it and read as usual
if r.BufferSize < 0 {
r.BufferSize = BufferDisable
r.buf = nil
r.pos = 0
return r.Reader.Read(p)
}
n, err = r.Reader.Read(p)
if len(r.buf)+n > r.BufferSize {
return 0, errors.New("probe reader overflow")
}
r.buf = append(r.buf, p[:n]...)
r.pos += n
return
}
func (r *ReadBuffer) Close() error {
if closer, ok := r.Reader.(io.Closer); ok {
return closer.Close()
}
return nil
}
func (r *ReadBuffer) Seek(offset int64, whence int) (int64, error) {
var pos int
switch whence {
case io.SeekStart:
pos = int(offset)
case io.SeekCurrent:
pos = r.pos + int(offset)
case io.SeekEnd:
pos = len(r.buf) + int(offset)
}
// negative offset - empty buffer
if pos < 0 {
r.buf = nil
r.pos = 0
} else if pos >= len(r.buf) {
r.pos = len(r.buf)
} else {
r.pos = pos
}
return int64(r.pos), nil
}
func (r *ReadBuffer) Peek(n int) ([]byte, error) {
r.BufferSize = n
b := make([]byte, n)
if _, err := io.ReadAtLeast(r, b, n); err != nil {
return nil, err
}
r.Reset()
return b, nil
}
func (r *ReadBuffer) Reset() {
r.BufferSize = BufferDrainAndClear
r.pos = 0
}
@@ -0,0 +1,64 @@
package core
import (
"bytes"
"io"
"testing"
"github.com/stretchr/testify/require"
)
func TestReadSeeker(t *testing.T) {
b := []byte{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
buf := bytes.NewReader(b)
rd := NewReadBuffer(buf)
rd.BufferSize = ProbeSize
// 1. Read to buffer
b = make([]byte, 3)
n, err := rd.Read(b)
require.Nil(t, err)
require.Equal(t, []byte{0, 1, 2}, b[:n])
// 2. Seek to start
_, err = rd.Seek(0, io.SeekStart)
require.Nil(t, err)
// 3. Read from buffer
b = make([]byte, 2)
n, err = rd.Read(b)
require.Nil(t, err)
require.Equal(t, []byte{0, 1}, b[:n])
// 4. Read from buffer
n, err = rd.Read(b)
require.Nil(t, err)
require.Equal(t, []byte{2}, b[:n])
// 5. Read to buffer
n, err = rd.Read(b)
require.Nil(t, err)
require.Equal(t, []byte{3, 4}, b[:n])
// 6. Seek to start
_, err = rd.Seek(0, io.SeekStart)
require.Nil(t, err)
// 7. Disable buffer
rd.BufferSize = -1
// 8. Read from buffer
b = make([]byte, 10)
n, err = rd.Read(b)
require.Nil(t, err)
require.Equal(t, []byte{0, 1, 2, 3, 4}, b[:n])
// 9. Direct read
n, err = rd.Read(b)
require.Nil(t, err)
require.Equal(t, []byte{5, 6, 7, 8, 9}, b[:n])
// 10. Check buffer empty
require.Nil(t, rd.buf)
}
@@ -0,0 +1,43 @@
package core
// This code copied from go1.21 for backward support in go1.20.
// We need to support go1.20 for Windows 7
// Index returns the index of the first occurrence of v in s,
// or -1 if not present.
func Index[S ~[]E, E comparable](s S, v E) int {
for i := range s {
if v == s[i] {
return i
}
}
return -1
}
// Contains reports whether v is present in s.
func Contains[S ~[]E, E comparable](s S, v E) bool {
return Index(s, v) >= 0
}
type Ordered interface {
~int | ~int8 | ~int16 | ~int32 | ~int64 |
~uint | ~uint8 | ~uint16 | ~uint32 | ~uint64 | ~uintptr |
~float32 | ~float64 |
~string
}
// Max returns the maximal value in x. It panics if x is empty.
// For floating-point E, Max propagates NaNs (any NaN value in x
// forces the output to be NaN).
func Max[S ~[]E, E Ordered](x S) E {
if len(x) < 1 {
panic("slices.Max: empty list")
}
m := x[0]
for i := 1; i < len(x); i++ {
if x[i] > m {
m = x[i]
}
}
return m
}
+217
View File
@@ -0,0 +1,217 @@
package core
import (
"encoding/json"
"errors"
"github.com/pion/rtp"
)
var ErrCantGetTrack = errors.New("can't get track")
type Receiver struct {
Node
// Deprecated: should be removed
Media *Media `json:"-"`
// Deprecated: should be removed
ID byte `json:"-"` // Channel for RTSP, PayloadType for MPEG-TS
Bytes int `json:"bytes,omitempty"`
Packets int `json:"packets,omitempty"`
}
func NewReceiver(media *Media, codec *Codec) *Receiver {
r := &Receiver{
Node: Node{id: NewID(), Codec: codec},
Media: media,
}
r.Input = func(packet *Packet) {
r.Bytes += len(packet.Payload)
r.Packets++
for _, child := range r.childs {
child.Input(packet)
}
}
return r
}
// Deprecated: should be removed
func (r *Receiver) WriteRTP(packet *rtp.Packet) {
r.Input(packet)
}
// Deprecated: should be removed
func (r *Receiver) Senders() []*Sender {
if len(r.childs) > 0 {
return []*Sender{{}}
} else {
return nil
}
}
// Deprecated: should be removed
func (r *Receiver) Replace(target *Receiver) {
MoveNode(&target.Node, &r.Node)
}
func (r *Receiver) Close() {
r.Node.Close()
}
type Sender struct {
Node
// Deprecated:
Media *Media `json:"-"`
// Deprecated:
Handler HandlerFunc `json:"-"`
Bytes int `json:"bytes,omitempty"`
Packets int `json:"packets,omitempty"`
Drops int `json:"drops,omitempty"`
buf chan *Packet
done chan struct{}
}
func NewSender(media *Media, codec *Codec) *Sender {
var bufSize uint16
if GetKind(codec.Name) == KindVideo {
if codec.IsRTP() {
// in my tests 40Mbit/s 4K-video can generate up to 1500 items
// for the h264.RTPDepay => RTPPay queue
bufSize = 4096
} else {
bufSize = 64
}
} else {
bufSize = 128
}
buf := make(chan *Packet, bufSize)
s := &Sender{
Node: Node{id: NewID(), Codec: codec},
Media: media,
buf: buf,
}
s.Input = func(packet *Packet) {
s.mu.Lock()
// unblock write to nil chan - OK, write to closed chan - panic
select {
case s.buf <- packet:
s.Bytes += len(packet.Payload)
s.Packets++
default:
s.Drops++
}
s.mu.Unlock()
}
s.Output = func(packet *Packet) {
s.Handler(packet)
}
return s
}
// Deprecated: should be removed
func (s *Sender) HandleRTP(parent *Receiver) {
s.WithParent(parent)
s.Start()
}
// Deprecated: should be removed
func (s *Sender) Bind(parent *Receiver) {
s.WithParent(parent)
}
func (s *Sender) WithParent(parent *Receiver) *Sender {
s.Node.WithParent(&parent.Node)
return s
}
func (s *Sender) Start() {
s.mu.Lock()
defer s.mu.Unlock()
if s.buf == nil || s.done != nil {
return
}
s.done = make(chan struct{})
// pass buf directly so that it's impossible for buf to be nil
go func(buf chan *Packet) {
for packet := range buf {
s.Output(packet)
}
close(s.done)
}(s.buf)
}
func (s *Sender) Wait() {
if done := s.done; done != nil {
<-done
}
}
func (s *Sender) State() string {
if s.buf == nil {
return "closed"
}
if s.done == nil {
return "new"
}
return "connected"
}
func (s *Sender) Close() {
// close buffer if exists
s.mu.Lock()
if s.buf != nil {
close(s.buf) // exit from for range loop
s.buf = nil // prevent writing to closed chan
}
s.mu.Unlock()
s.Node.Close()
}
func (r *Receiver) MarshalJSON() ([]byte, error) {
v := struct {
ID uint32 `json:"id"`
Codec *Codec `json:"codec"`
Childs []uint32 `json:"childs,omitempty"`
Bytes int `json:"bytes,omitempty"`
Packets int `json:"packets,omitempty"`
}{
ID: r.Node.id,
Codec: r.Node.Codec,
Bytes: r.Bytes,
Packets: r.Packets,
}
for _, child := range r.childs {
v.Childs = append(v.Childs, child.id)
}
return json.Marshal(v)
}
func (s *Sender) MarshalJSON() ([]byte, error) {
v := struct {
ID uint32 `json:"id"`
Codec *Codec `json:"codec"`
Parent uint32 `json:"parent,omitempty"`
Bytes int `json:"bytes,omitempty"`
Packets int `json:"packets,omitempty"`
Drops int `json:"drops,omitempty"`
}{
ID: s.Node.id,
Codec: s.Node.Codec,
Bytes: s.Bytes,
Packets: s.Packets,
Drops: s.Drops,
}
if s.parent != nil {
v.Parent = s.parent.id
}
return json.Marshal(v)
}
@@ -0,0 +1,53 @@
package core
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestSenser(t *testing.T) {
recv := make(chan *Packet) // blocking receiver
sender := NewSender(nil, &Codec{})
sender.Output = func(packet *Packet) {
recv <- packet
}
require.Equal(t, "new", sender.State())
sender.Start()
require.Equal(t, "connected", sender.State())
sender.Input(&Packet{})
sender.Input(&Packet{})
require.Equal(t, 2, sender.Packets)
require.Equal(t, 0, sender.Drops)
// important to read one before close
// because goroutine in Start() can run with nil chan
// it's OK in real life, but bad for test
_, ok := <-recv
require.True(t, ok)
sender.Close()
require.Equal(t, "closed", sender.State())
sender.Input(&Packet{})
require.Equal(t, 2, sender.Packets)
require.Equal(t, 1, sender.Drops)
// read 2nd
_, ok = <-recv
require.True(t, ok)
// read 3rd
select {
case <-recv:
ok = true
default:
ok = false
}
require.False(t, ok)
}
@@ -0,0 +1,74 @@
package core
import (
"sync"
)
// Waiter support:
// - autotart on first Wait
// - block new waiters after last Done
// - safe Done after finish
type Waiter struct {
sync.WaitGroup
mu sync.Mutex
state int // state < 0 means finish
err error
}
func (w *Waiter) Add(delta int) {
w.mu.Lock()
if w.state >= 0 {
w.state += delta
w.WaitGroup.Add(delta)
}
w.mu.Unlock()
}
func (w *Waiter) Wait() error {
w.mu.Lock()
// first wait auto start waiter
if w.state == 0 {
w.state++
w.WaitGroup.Add(1)
}
w.mu.Unlock()
w.WaitGroup.Wait()
return w.err
}
func (w *Waiter) Done(err error) {
w.mu.Lock()
// safe run Done only when have tasks
if w.state > 0 {
w.state--
w.WaitGroup.Done()
}
// block waiter for any operations after last done
if w.state == 0 {
w.state = -1
w.err = err
}
w.mu.Unlock()
}
func (w *Waiter) WaitChan() <-chan error {
var ch chan error
w.mu.Lock()
if w.state >= 0 {
ch = make(chan error)
go func() {
ch <- w.Wait()
}()
}
w.mu.Unlock()
return ch
}
@@ -0,0 +1,52 @@
package core
import (
"time"
)
type Worker struct {
timer *time.Timer
done chan struct{}
}
// NewWorker run f after d
func NewWorker(d time.Duration, f func() time.Duration) *Worker {
timer := time.NewTimer(d)
done := make(chan struct{})
go func() {
for {
select {
case <-timer.C:
if d = f(); d > 0 {
timer.Reset(d)
continue
}
case <-done:
timer.Stop()
}
break
}
}()
return &Worker{timer: timer, done: done}
}
// Do - instant timer run
func (w *Worker) Do() {
if w == nil {
return
}
w.timer.Reset(0)
}
func (w *Worker) Stop() {
if w == nil {
return
}
select {
case w.done <- struct{}{}:
default:
}
}
@@ -0,0 +1,114 @@
package core
import (
"bytes"
"io"
"net/http"
"sync"
)
// WriteBuffer by defaul Write(s) to bytes.Buffer.
// But after WriteTo to new io.Writer - calls Reset.
// Reset will flush current buffer data to new writer and starts to Write to new io.Writer
// WriteTo will be locked until Write fails or Close will be called.
type WriteBuffer struct {
io.Writer
err error
mu sync.Mutex
wg sync.WaitGroup
state byte
}
func NewWriteBuffer(wr io.Writer) *WriteBuffer {
if wr == nil {
wr = bytes.NewBuffer(nil)
}
return &WriteBuffer{Writer: wr}
}
func (w *WriteBuffer) Write(p []byte) (n int, err error) {
w.mu.Lock()
if w.err != nil {
err = w.err
} else if n, err = w.Writer.Write(p); err != nil {
w.err = err
w.done()
} else if f, ok := w.Writer.(http.Flusher); ok {
f.Flush()
}
w.mu.Unlock()
return
}
func (w *WriteBuffer) WriteTo(wr io.Writer) (n int64, err error) {
w.Reset(wr)
w.wg.Wait()
return 0, w.err // TODO: fix counter
}
func (w *WriteBuffer) Close() error {
if closer, ok := w.Writer.(io.Closer); ok {
return closer.Close()
}
w.mu.Lock()
w.done()
w.mu.Unlock()
return nil
}
func (w *WriteBuffer) Reset(wr io.Writer) {
w.mu.Lock()
w.add()
if buf, ok := w.Writer.(*bytes.Buffer); ok && buf.Len() != 0 {
if _, err := io.Copy(wr, buf); err != nil {
w.err = err
w.done()
}
}
w.Writer = wr
w.mu.Unlock()
}
const (
none = iota
start
end
)
func (w *WriteBuffer) add() {
if w.state == none {
w.state = start
w.wg.Add(1)
}
}
func (w *WriteBuffer) done() {
if w.state == start {
w.state = end
w.wg.Done()
}
}
// OnceBuffer will catch only first message
type OnceBuffer struct {
buf []byte
}
func (o *OnceBuffer) Write(p []byte) (n int, err error) {
if o.buf == nil {
o.buf = p
}
return 0, io.EOF
}
func (o *OnceBuffer) WriteTo(w io.Writer) (n int64, err error) {
return io.Copy(w, bytes.NewReader(o.buf))
}
func (o *OnceBuffer) Buffer() []byte {
return o.buf
}
func (o *OnceBuffer) Len() int {
return len(o.buf)
}
@@ -0,0 +1,7 @@
# Credentials
This module allows you to get variables:
- from custom storage (ex. config file)
- from [credential files](https://systemd.io/CREDENTIALS/)
- from environment variables
@@ -0,0 +1,79 @@
package creds
import (
"errors"
"os"
"path/filepath"
"regexp"
"strings"
)
type Storage interface {
SetValue(name, value string) error
GetValue(name string) (string, bool)
}
var storage Storage
func SetStorage(s Storage) {
storage = s
}
func SetValue(name, value string) error {
if storage == nil {
return errors.New("credentials: storage not initialized")
}
if err := storage.SetValue(name, value); err != nil {
return err
}
AddSecret(value)
return nil
}
func GetValue(name string) (value string, ok bool) {
value, ok = getValue(name)
AddSecret(value)
return
}
func getValue(name string) (string, bool) {
if storage != nil {
if value, ok := storage.GetValue(name); ok {
return value, true
}
}
if dir, ok := os.LookupEnv("CREDENTIALS_DIRECTORY"); ok {
if value, _ := os.ReadFile(filepath.Join(dir, name)); value != nil {
return strings.TrimSpace(string(value)), true
}
}
return os.LookupEnv(name)
}
// ReplaceVars - support format ${CAMERA_PASSWORD} and ${RTSP_USER:admin}
func ReplaceVars(data []byte) []byte {
re := regexp.MustCompile(`\${([^}{]+)}`)
return re.ReplaceAllFunc(data, func(match []byte) []byte {
key := string(match[2 : len(match)-1])
var def string
var defok bool
if i := strings.IndexByte(key, ':'); i > 0 {
key, def = key[:i], key[i+1:]
defok = true
}
if value, ok := GetValue(key); ok {
return []byte(value)
}
if defok {
return []byte(def)
}
return match
})
}
@@ -0,0 +1,94 @@
package creds
import (
"io"
"net/http"
"regexp"
"slices"
"strings"
"sync"
)
func AddSecret(value string) {
if value == "" {
return
}
secretsMu.Lock()
defer secretsMu.Unlock()
if slices.Contains(secrets, value) {
return
}
secrets = append(secrets, value)
secretsReplacer = nil
}
var secrets []string
var secretsMu sync.Mutex
var secretsReplacer *strings.Replacer
var userinfoRegexp *regexp.Regexp
func getReplacer() *strings.Replacer {
secretsMu.Lock()
defer secretsMu.Unlock()
if secretsReplacer == nil {
oldnew := make([]string, 0, 2*len(secrets))
for _, s := range secrets {
oldnew = append(oldnew, s, "***")
}
secretsReplacer = strings.NewReplacer(oldnew...)
}
if userinfoRegexp == nil {
userinfoRegexp = regexp.MustCompile(`://[` + userinfo + `]+@`)
}
return secretsReplacer
}
// Uniform Resource Identifier (URI)
// https://datatracker.ietf.org/doc/html/rfc3986
const (
unreserved = `A-Za-z0-9-._~`
subdelims = `!$&'()*+,;=`
userinfo = unreserved + subdelims + `%:`
)
func SecretString(s string) string {
re := getReplacer()
s = userinfoRegexp.ReplaceAllString(s, `://***@`)
return re.Replace(s)
}
func SecretWrite(w io.Writer, s string) (n int, err error) {
re := getReplacer()
s = userinfoRegexp.ReplaceAllString(s, `://***@`)
return re.WriteString(w, s)
}
func SecretWriter(w io.Writer) io.Writer {
return &secretWriter{w}
}
type secretWriter struct {
w io.Writer
}
func (s *secretWriter) Write(b []byte) (int, error) {
return SecretWrite(s.w, string(b))
}
func SecretResponse(w http.ResponseWriter) http.ResponseWriter {
return &secretResponse{w}
}
type secretResponse struct {
http.ResponseWriter
}
func (s *secretResponse) Write(b []byte) (int, error) {
return SecretWrite(s.ResponseWriter, string(b))
}
@@ -0,0 +1,15 @@
package creds
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestString(t *testing.T) {
AddSecret("admin")
AddSecret("pa$$word")
s := SecretString("rtsp://admin:pa$$word@192.168.1.123/stream1")
require.Equal(t, "rtsp://***:***@192.168.1.123/stream1", s)
}
+47
View File
@@ -0,0 +1,47 @@
package debug
import (
"bytes"
"math/rand"
"net"
)
type badConn struct {
net.Conn
delay int
buf []byte
}
func NewBadConn(conn net.Conn) net.Conn {
return &badConn{Conn: conn}
}
const (
missChance = 0.05
delayChance = 0.1
)
func (c *badConn) Read(b []byte) (n int, err error) {
if rand.Float32() < missChance {
if _, err = c.Conn.Read(b); err != nil {
return
}
//log.Printf("bad conn: miss")
}
if c.delay > 0 {
if c.delay--; c.delay == 0 {
n = copy(b, c.buf)
return
}
} else if rand.Float32() < delayChance {
if n, err = c.Conn.Read(b); err != nil {
return
}
c.delay = 1 + rand.Intn(5)
c.buf = bytes.Clone(b[:n])
//log.Printf("bad conn: delay %d", c.delay)
}
return c.Conn.Read(b)
}
@@ -0,0 +1,58 @@
package debug
import (
"fmt"
"time"
"github.com/pion/rtp"
)
func Logger(include func(packet *rtp.Packet) bool) func(packet *rtp.Packet) {
var lastTime = time.Now()
var lastTS uint32
var secCnt int
var secSize int
var secTS uint32
var secTime time.Time
return func(packet *rtp.Packet) {
if include != nil && !include(packet) {
return
}
now := time.Now()
fmt.Printf(
"%s: size=%6d ts=%10d type=%2d ssrc=%d seq=%5d mark=%t dts=%4d dtime=%3dms\n",
now.Format("15:04:05.000"),
len(packet.Payload), packet.Timestamp, packet.PayloadType, packet.SSRC, packet.SequenceNumber, packet.Marker,
packet.Timestamp-lastTS, now.Sub(lastTime).Milliseconds(),
)
lastTS = packet.Timestamp
lastTime = now
if secTS == 0 {
secTS = lastTS
secTime = now
return
}
if dt := now.Sub(secTime); dt > time.Second {
fmt.Printf(
"%s: size=%6d cnt=%d dts=%d dtime=%3dms\n",
now.Format("15:04:05.000"),
secSize, secCnt, lastTS-secTS, dt.Milliseconds(),
)
secCnt = 0
secSize = 0
secTS = lastTS
secTime = now
}
secCnt++
secSize += len(packet.Payload)
}
}
@@ -0,0 +1,95 @@
package doorbird
import (
"fmt"
"net"
"net/url"
"time"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/pion/rtp"
)
type Client struct {
core.Connection
conn net.Conn
}
func Dial(rawURL string) (*Client, error) {
u, err := url.Parse(rawURL)
if err != nil {
return nil, err
}
user := u.User.Username()
pass, _ := u.User.Password()
if u.Port() == "" {
u.Host += ":80"
}
conn, err := net.DialTimeout("tcp", u.Host, core.ConnDialTimeout)
if err != nil {
return nil, err
}
s := fmt.Sprintf("POST /bha-api/audio-transmit.cgi?http-user=%s&http-password=%s HTTP/1.0\r\n", user, pass) +
"Content-Type: audio/basic\r\n" +
"Content-Length: 9999999\r\n" +
"Connection: Keep-Alive\r\n" +
"Cache-Control: no-cache\r\n" +
"\r\n"
_ = conn.SetWriteDeadline(time.Now().Add(core.ConnDeadline))
if _, err = conn.Write([]byte(s)); err != nil {
return nil, err
}
medias := []*core.Media{
{
Kind: core.KindAudio,
Direction: core.DirectionSendonly,
Codecs: []*core.Codec{
{Name: core.CodecPCMU, ClockRate: 8000},
},
},
}
return &Client{
core.Connection{
ID: core.NewID(),
FormatName: "doorbird",
Protocol: "http",
URL: rawURL,
Medias: medias,
Transport: conn,
},
conn,
}, nil
}
func (c *Client) GetTrack(media *core.Media, codec *core.Codec) (*core.Receiver, error) {
return nil, core.ErrCantGetTrack
}
func (c *Client) AddTrack(media *core.Media, codec *core.Codec, track *core.Receiver) error {
sender := core.NewSender(media, track.Codec)
sender.Handler = func(pkt *rtp.Packet) {
_ = c.conn.SetWriteDeadline(time.Now().Add(core.ConnDeadline))
if n, err := c.conn.Write(pkt.Payload); err == nil {
c.Send += n
}
}
sender.HandleRTP(track)
c.Senders = append(c.Senders, sender)
return nil
}
func (c *Client) Start() (err error) {
// just block until c.conn closed
b := make([]byte, 1)
_, err = c.conn.Read(b)
return
}
@@ -0,0 +1,79 @@
package dvrip
import (
"encoding/binary"
"time"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/pion/rtp"
)
type Backchannel struct {
core.Connection
client *Client
}
func (c *Backchannel) GetTrack(media *core.Media, codec *core.Codec) (*core.Receiver, error) {
return nil, core.ErrCantGetTrack
}
func (c *Backchannel) Start() error {
if err := c.client.conn.SetReadDeadline(time.Time{}); err != nil {
return err
}
b := make([]byte, 4096)
for {
if _, err := c.client.rd.Read(b); err != nil {
return err
}
}
}
func (c *Backchannel) AddTrack(media *core.Media, _ *core.Codec, track *core.Receiver) error {
if err := c.client.Talk(); err != nil {
return err
}
const PacketSize = 320
buf := make([]byte, 8+PacketSize)
binary.BigEndian.PutUint32(buf, 0x1FA)
switch track.Codec.Name {
case core.CodecPCMU:
buf[4] = 10
case core.CodecPCMA:
buf[4] = 14
}
//for i, rate := range sampleRates {
// if rate == track.Codec.ClockRate {
// buf[5] = byte(i) + 1
// break
// }
//}
buf[5] = 2 // ClockRate=8000
binary.LittleEndian.PutUint16(buf[6:], PacketSize)
var payload []byte
sender := core.NewSender(media, track.Codec)
sender.Handler = func(packet *rtp.Packet) {
payload = append(payload, packet.Payload...)
for len(payload) >= PacketSize {
buf = append(buf[:8], payload[:PacketSize]...)
if n, err := c.client.WriteCmd(OPTalkData, buf); err != nil {
c.Send += n
}
payload = payload[PacketSize:]
}
}
sender.HandleRTP(track)
c.Senders = append(c.Senders, sender)
return nil
}
+247
View File
@@ -0,0 +1,247 @@
package dvrip
import (
"bufio"
"bytes"
"crypto/md5"
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/url"
"time"
)
const (
Login = 1000
OPMonitorClaim = 1413
OPMonitorStart = 1410
OPTalkClaim = 1434
OPTalkStart = 1430
OPTalkData = 1432
)
type Client struct {
conn net.Conn
session uint32
seq uint32
stream string
rd io.Reader
buf []byte
}
func (c *Client) Dial(rawURL string) (err error) {
u, err := url.Parse(rawURL)
if err != nil {
return
}
if u.Port() == "" {
// add default TCP port
u.Host += ":34567"
}
c.conn, err = net.DialTimeout("tcp", u.Host, time.Second*3)
if err != nil {
return
}
if query := u.Query(); query.Get("backchannel") != "1" {
channel := query.Get("channel")
if channel == "" {
channel = "0"
}
subtype := query.Get("subtype")
switch subtype {
case "", "0":
subtype = "Main"
case "1":
subtype = "Extra1"
}
c.stream = fmt.Sprintf(
`{"Channel":%s,"CombinMode":"NONE","StreamType":"%s","TransMode":"TCP"}`,
channel, subtype,
)
}
c.rd = bufio.NewReader(c.conn)
if u.User != nil {
pass, _ := u.User.Password()
return c.Login(u.User.Username(), pass)
} else {
return c.Login("admin", "admin")
}
}
func (c *Client) Close() error {
return c.conn.Close()
}
func (c *Client) Login(user, pass string) (err error) {
data := fmt.Sprintf(
`{"EncryptType":"MD5","LoginType":"DVRIP-Web","PassWord":"%s","UserName":"%s"}`+"\x0A\x00",
SofiaHash(pass), user,
)
if _, err = c.WriteCmd(Login, []byte(data)); err != nil {
return
}
_, err = c.ReadJSON()
return
}
func (c *Client) Play() error {
format := `{"Name":"OPMonitor","SessionID":"0x%08X","OPMonitor":{"Action":"%s","Parameter":%s}}` + "\x0A\x00"
data := fmt.Sprintf(format, c.session, "Claim", c.stream)
if _, err := c.WriteCmd(OPMonitorClaim, []byte(data)); err != nil {
return err
}
if _, err := c.ReadJSON(); err != nil {
return err
}
data = fmt.Sprintf(format, c.session, "Start", c.stream)
_, err := c.WriteCmd(OPMonitorStart, []byte(data))
return err
}
func (c *Client) Talk() error {
format := `{"Name":"OPTalk","SessionID":"0x%08X","OPTalk":{"Action":"%s","AudioFormat":{"EncodeType":"G711_ALAW"}}}` + "\x0A\x00"
data := fmt.Sprintf(format, c.session, "Claim")
if _, err := c.WriteCmd(OPTalkClaim, []byte(data)); err != nil {
return err
}
if _, err := c.ReadJSON(); err != nil {
return err
}
data = fmt.Sprintf(format, c.session, "Start")
_, err := c.WriteCmd(OPTalkStart, []byte(data))
return err
}
func (c *Client) WriteCmd(cmd uint16, payload []byte) (n int, err error) {
b := make([]byte, 20, 128)
b[0] = 255
binary.LittleEndian.PutUint32(b[4:], c.session)
binary.LittleEndian.PutUint32(b[8:], c.seq)
binary.LittleEndian.PutUint16(b[14:], cmd)
binary.LittleEndian.PutUint32(b[16:], uint32(len(payload)))
b = append(b, payload...)
c.seq++
if err = c.conn.SetWriteDeadline(time.Now().Add(time.Second * 5)); err != nil {
return 0, err
}
return c.conn.Write(b)
}
func (c *Client) ReadChunk() (b []byte, err error) {
if err = c.conn.SetReadDeadline(time.Now().Add(time.Second * 5)); err != nil {
return
}
b = make([]byte, 20)
if _, err = io.ReadFull(c.rd, b); err != nil {
return
}
if b[0] != 255 {
return nil, errors.New("read error")
}
c.session = binary.LittleEndian.Uint32(b[4:])
size := binary.LittleEndian.Uint32(b[16:])
b = make([]byte, size)
if _, err = io.ReadFull(c.rd, b); err != nil {
return
}
return
}
func (c *Client) ReadPacket() (pType byte, payload []byte, err error) {
var b []byte
// many cameras may split packet to multiple chunks
// some rare cameras may put multiple packets to single chunk
for len(c.buf) < 16 {
if b, err = c.ReadChunk(); err != nil {
return 0, nil, err
}
c.buf = append(c.buf, b...)
}
if !bytes.HasPrefix(c.buf, []byte{0, 0, 1}) {
return 0, nil, fmt.Errorf("dvrip: wrong packet: %0.16x", c.buf)
}
var size int
switch pType = c.buf[3]; pType {
case 0xFC, 0xFE:
size = int(binary.LittleEndian.Uint32(c.buf[12:])) + 16
case 0xFD: // PFrame
size = int(binary.LittleEndian.Uint32(c.buf[4:])) + 8
case 0xFA, 0xF9:
size = int(binary.LittleEndian.Uint16(c.buf[6:])) + 8
default:
return 0, nil, fmt.Errorf("dvrip: unknown packet type: %X", pType)
}
for len(c.buf) < size {
if b, err = c.ReadChunk(); err != nil {
return 0, nil, err
}
c.buf = append(c.buf, b...)
}
payload = c.buf[:size]
c.buf = c.buf[size:]
return
}
type Response map[string]any
func (c *Client) ReadJSON() (res Response, err error) {
b, err := c.ReadChunk()
if err != nil {
return
}
res = Response{}
if err = json.Unmarshal(b[:len(b)-2], &res); err != nil {
return
}
if v, ok := res["Ret"].(float64); !ok || (v != 100 && v != 515) {
err = fmt.Errorf("wrong response: %s", b)
}
return
}
func SofiaHash(password string) string {
const chars = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
sofia := make([]byte, 0, 8)
hash := md5.Sum([]byte(password))
for i := 0; i < md5.Size; i += 2 {
j := uint16(hash[i]) + uint16(hash[i+1])
sofia = append(sofia, chars[j%62])
}
return string(sofia)
}
@@ -0,0 +1,39 @@
package dvrip
import "github.com/AlexxIT/go2rtc/pkg/core"
func Dial(url string) (core.Producer, error) {
client := &Client{}
if err := client.Dial(url); err != nil {
return nil, err
}
conn := core.Connection{
ID: core.NewID(),
FormatName: "dvrip",
Protocol: "tcp",
RemoteAddr: client.conn.RemoteAddr().String(),
Transport: client.conn,
}
if client.stream != "" {
prod := &Producer{Connection: conn, client: client}
if err := prod.probe(); err != nil {
return nil, err
}
return prod, nil
} else {
conn.Medias = []*core.Media{
{
Kind: core.KindAudio,
Direction: core.DirectionSendonly,
Codecs: []*core.Codec{
// leave only one codec here for better compatibility with cameras
// https://github.com/AlexxIT/go2rtc/issues/1111
{Name: core.CodecPCMA, ClockRate: 8000, PayloadType: 8},
},
},
}
return &Backchannel{Connection: conn, client: client}, nil
}
}
@@ -0,0 +1,262 @@
package dvrip
import (
"encoding/base64"
"encoding/binary"
"errors"
"fmt"
"time"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/h264"
"github.com/AlexxIT/go2rtc/pkg/h264/annexb"
"github.com/AlexxIT/go2rtc/pkg/h265"
"github.com/pion/rtp"
)
type Producer struct {
core.Connection
client *Client
video, audio *core.Receiver
videoTS uint32
videoDT uint32
audioTS uint32
audioSeq uint16
}
func (c *Producer) Start() error {
for {
pType, b, err := c.client.ReadPacket()
if err != nil {
return err
}
//log.Printf("[DVR] type: %d, len: %d", dataType, len(b))
switch pType {
case 0xFC, 0xFE, 0xFD:
if c.video == nil {
continue
}
var payload []byte
if pType != 0xFD {
payload = b[16:] // iframe
} else {
payload = b[8:] // pframe
}
c.videoTS += c.videoDT
packet := &rtp.Packet{
Header: rtp.Header{Timestamp: c.videoTS},
Payload: annexb.EncodeToAVCC(payload),
}
//log.Printf("[AVC] %v, len: %d, ts: %10d", h265.Types(payload), len(payload), packet.Timestamp)
c.video.WriteRTP(packet)
case 0xFA: // audio
if c.audio == nil {
continue
}
payload := b[8:]
c.audioTS += uint32(len(payload))
c.audioSeq++
packet := &rtp.Packet{
Header: rtp.Header{
Version: 2,
Marker: true,
SequenceNumber: c.audioSeq,
Timestamp: c.audioTS,
},
Payload: payload,
}
//log.Printf("[DVR] len: %d, ts: %10d", len(packet.Payload), packet.Timestamp)
c.audio.WriteRTP(packet)
case 0xF9: // unknown
default:
println(fmt.Sprintf("dvrip: unknown packet type: %d", pType))
}
}
}
func (c *Producer) probe() error {
if err := c.client.Play(); err != nil {
return err
}
rd := core.NewReadBuffer(c.client.rd)
rd.BufferSize = core.ProbeSize
defer func() {
c.client.buf = nil
rd.Reset()
}()
c.client.rd = rd
// some awful cameras has VERY rare keyframes
// so we wait video+audio for default probe time
// and wait anything for 15 seconds
timeoutBoth := time.Now().Add(core.ProbeTimeout)
timeoutAny := time.Now().Add(time.Second * 15)
for {
if now := time.Now(); now.Before(timeoutBoth) {
if c.video != nil && c.audio != nil {
return nil
}
} else if now.Before(timeoutAny) {
if c.video != nil || c.audio != nil {
return nil
}
} else {
return errors.New("dvrip: can't probe medias")
}
tag, b, err := c.client.ReadPacket()
if err != nil {
return err
}
switch tag {
case 0xFC, 0xFE: // video
if c.video != nil {
continue
}
fps := b[5]
//width := uint16(b[6]) * 8
//height := uint16(b[7]) * 8
//println(width, height)
ts := b[8:]
// the exact value of the start TS does not matter
c.videoTS = binary.LittleEndian.Uint32(ts)
c.videoDT = 90000 / uint32(fps)
payload := annexb.EncodeToAVCC(b[16:])
c.addVideoTrack(b[4], payload)
case 0xFA: // audio
if c.audio != nil {
continue
}
// the exact value of the start TS does not matter
c.audioTS = c.videoTS
c.addAudioTrack(b[4], b[5])
}
}
}
func (c *Producer) addVideoTrack(mediaCode byte, payload []byte) {
var codec *core.Codec
switch mediaCode {
case 0x02, 0x12:
codec = &core.Codec{
Name: core.CodecH264,
ClockRate: 90000,
PayloadType: core.PayloadTypeRAW,
FmtpLine: h264.GetFmtpLine(payload),
}
case 0x03, 0x13, 0x43, 0x53:
codec = &core.Codec{
Name: core.CodecH265,
ClockRate: 90000,
PayloadType: core.PayloadTypeRAW,
FmtpLine: "profile-id=1",
}
for {
size := 4 + int(binary.BigEndian.Uint32(payload))
switch h265.NALUType(payload) {
case h265.NALUTypeVPS:
codec.FmtpLine += ";sprop-vps=" + base64.StdEncoding.EncodeToString(payload[4:size])
case h265.NALUTypeSPS:
codec.FmtpLine += ";sprop-sps=" + base64.StdEncoding.EncodeToString(payload[4:size])
case h265.NALUTypePPS:
codec.FmtpLine += ";sprop-pps=" + base64.StdEncoding.EncodeToString(payload[4:size])
}
if size < len(payload) {
payload = payload[size:]
} else {
break
}
}
default:
println("[DVRIP] unsupported video codec:", mediaCode)
return
}
media := &core.Media{
Kind: core.KindVideo,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{codec},
}
c.Medias = append(c.Medias, media)
c.video = core.NewReceiver(media, codec)
c.Receivers = append(c.Receivers, c.video)
}
var sampleRates = []uint32{4000, 8000, 11025, 16000, 20000, 22050, 32000, 44100, 48000}
func (c *Producer) addAudioTrack(mediaCode byte, sampleRate byte) {
// https://github.com/vigoss30611/buildroot-ltc/blob/master/system/qm/ipc/ProtocolService/src/ZhiNuo/inc/zn_dh_base_type.h
// PCM8 = 7, G729, IMA_ADPCM, G711U, G721, PCM8_VWIS, MS_ADPCM, G711A, PCM16
var codec *core.Codec
switch mediaCode {
case 10: // G711U
codec = &core.Codec{
Name: core.CodecPCMU,
}
case 14: // G711A
codec = &core.Codec{
Name: core.CodecPCMA,
}
default:
println("[DVRIP] unsupported audio codec:", mediaCode)
return
}
if sampleRate <= byte(len(sampleRates)) {
codec.ClockRate = sampleRates[sampleRate-1]
}
media := &core.Media{
Kind: core.KindAudio,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{codec},
}
c.Medias = append(c.Medias, media)
c.audio = core.NewReceiver(media, codec)
c.Receivers = append(c.Receivers, c.audio)
}
//func (c *Client) MarshalJSON() ([]byte, error) {
// info := &core.Info{
// Type: "DVRIP active producer",
// RemoteAddr: c.conn.RemoteAddr().String(),
// Medias: c.Medias,
// Receivers: c.Receivers,
// Recv: c.Recv,
// }
// return json.Marshal(info)
//}
@@ -0,0 +1,180 @@
package eseecloud
import (
"bytes"
"encoding/binary"
"errors"
"io"
"net/http"
"regexp"
"strings"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/h264/annexb"
"github.com/pion/rtp"
)
type Producer struct {
core.Connection
rd *core.ReadBuffer
videoPT, audioPT uint8
}
func Dial(rawURL string) (core.Producer, error) {
rawURL, _ = strings.CutPrefix(rawURL, "eseecloud")
res, err := http.Get("http" + rawURL)
if err != nil {
return nil, err
}
prod, err := Open(res.Body)
if err != nil {
return nil, err
}
if info, ok := prod.(core.Info); ok {
info.SetProtocol("http")
info.SetURL(rawURL)
}
return prod, nil
}
func Open(r io.Reader) (core.Producer, error) {
prod := &Producer{
Connection: core.Connection{
ID: core.NewID(),
FormatName: "eseecloud",
Transport: r,
},
rd: core.NewReadBuffer(r),
}
if err := prod.probe(); err != nil {
return nil, err
}
return prod, nil
}
func (p *Producer) probe() error {
b, err := p.rd.Peek(1024)
if err != nil {
return err
}
i := bytes.Index(b, []byte("\r\n\r\n"))
if i == -1 {
return io.EOF
}
b = make([]byte, i+4)
_, _ = p.rd.Read(b)
re := regexp.MustCompile(`m=(video|audio) (\d+) (\w+)/(\d+)\S*`)
for _, item := range re.FindAllStringSubmatch(string(b), 2) {
p.SDP += item[0] + "\n"
switch item[3] {
case "H264", "H265":
p.Medias = append(p.Medias, &core.Media{
Kind: core.KindVideo,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{
{
Name: item[3],
ClockRate: 90000,
PayloadType: core.PayloadTypeRAW,
},
},
})
p.videoPT = byte(core.Atoi(item[2]))
case "G711":
p.Medias = append(p.Medias, &core.Media{
Kind: core.KindAudio,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{
{
Name: core.CodecPCMA,
ClockRate: 8000,
},
},
})
p.audioPT = byte(core.Atoi(item[2]))
}
}
return nil
}
func (p *Producer) Start() error {
receivers := make(map[uint8]*core.Receiver)
for _, receiver := range p.Receivers {
switch receiver.Codec.Kind() {
case core.KindVideo:
receivers[p.videoPT] = receiver
case core.KindAudio:
receivers[p.audioPT] = receiver
}
}
for {
pkt, err := p.readPacket()
if err != nil {
return err
}
if recv := receivers[pkt.PayloadType]; recv != nil {
switch recv.Codec.Name {
case core.CodecH264, core.CodecH265:
// timestamp = seconds x 1000000
pkt = &rtp.Packet{
Header: rtp.Header{
Timestamp: uint32(uint64(pkt.Timestamp) * 90000 / 1000000),
},
Payload: annexb.EncodeToAVCC(pkt.Payload),
}
case core.CodecPCMA:
pkt = &rtp.Packet{
Header: rtp.Header{
Version: 2,
SequenceNumber: pkt.SequenceNumber,
Timestamp: uint32(uint64(pkt.Timestamp) * 8000 / 1000000),
},
Payload: pkt.Payload,
}
}
recv.WriteRTP(pkt)
}
}
}
func (p *Producer) readPacket() (*core.Packet, error) {
b := make([]byte, 8)
if _, err := io.ReadFull(p.rd, b); err != nil {
return nil, err
}
if b[0] != '$' {
return nil, errors.New("eseecloud: wrong start byte")
}
size := binary.BigEndian.Uint32(b[4:])
b = make([]byte, size)
if _, err := io.ReadFull(p.rd, b); err != nil {
return nil, err
}
pkt := &core.Packet{}
if err := pkt.Unmarshal(b); err != nil {
return nil, err
}
p.Recv += int(size)
return pkt, nil
}
+166
View File
@@ -0,0 +1,166 @@
package expr
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/cookiejar"
"net/url"
"regexp"
"strings"
"time"
"github.com/expr-lang/expr"
"github.com/expr-lang/expr/vm"
)
func newRequest(rawURL string, options map[string]any) (*http.Request, error) {
var method, contentType string
var rd io.Reader
// method from js fetch
if s, ok := options["method"].(string); ok {
method = s
} else {
method = "GET"
}
// params key from python requests
if kv, ok := options["params"].(map[string]any); ok {
rawURL += "?" + url.Values(kvToString(kv)).Encode()
}
// json key from python requests
// data key from python requests
// body key from js fetch
if v, ok := options["json"]; ok {
b, err := json.Marshal(v)
if err != nil {
return nil, err
}
contentType = "application/json"
rd = bytes.NewReader(b)
} else if kv, ok := options["data"].(map[string]any); ok {
contentType = "application/x-www-form-urlencoded"
rd = strings.NewReader(url.Values(kvToString(kv)).Encode())
} else if s, ok := options["body"].(string); ok {
rd = strings.NewReader(s)
}
req, err := http.NewRequest(method, rawURL, rd)
if err != nil {
return nil, err
}
if kv, ok := options["headers"].(map[string]any); ok {
req.Header = kvToString(kv)
}
if contentType != "" && req.Header.Get("Content-Type") == "" {
req.Header.Set("Content-Type", contentType)
}
return req, nil
}
func kvToString(kv map[string]any) map[string][]string {
dst := make(map[string][]string, len(kv))
for k, v := range kv {
dst[k] = []string{fmt.Sprintf("%v", v)}
}
return dst
}
func regExp(params ...any) (*regexp.Regexp, error) {
exp := params[0].(string)
if len(params) >= 2 {
// support:
// i case-insensitive (default false)
// m multi-line mode: ^ and $ match begin/end line (default false)
// s let . match \n (default false)
// https://pkg.go.dev/regexp/syntax
flags := params[1].(string)
exp = "(?" + flags + ")" + exp
}
return regexp.Compile(exp)
}
func Compile(input string) (*vm.Program, error) {
// support http sessions
jar, _ := cookiejar.New(nil)
client := http.Client{
Jar: jar,
Timeout: 5 * time.Second,
}
return expr.Compile(
input,
expr.Function(
"fetch",
func(params ...any) (any, error) {
var req *http.Request
var err error
rawURL := params[0].(string)
if len(params) == 2 {
options := params[1].(map[string]any)
req, err = newRequest(rawURL, options)
} else {
req, err = http.NewRequest("GET", rawURL, nil)
}
if err != nil {
return nil, err
}
res, err := client.Do(req)
if err != nil {
return nil, err
}
b, _ := io.ReadAll(res.Body)
return map[string]any{
"ok": res.StatusCode < 400,
"status": res.Status,
"text": string(b),
"json": func() (v any) {
_ = json.Unmarshal(b, &v)
return
},
}, nil
},
//new(func(url string) map[string]any),
//new(func(url string, options map[string]any) map[string]any),
),
expr.Function(
"match",
func(params ...any) (any, error) {
re, err := regExp(params[1:]...)
if err != nil {
return nil, err
}
str := params[0].(string)
return re.FindStringSubmatch(str), nil
},
//new(func(str, expr string) []string),
//new(func(str, expr, flags string) []string),
),
)
}
func Eval(input string, env any) (any, error) {
program, err := Compile(input)
if err != nil {
return nil, err
}
return expr.Run(program, env)
}
func Run(program *vm.Program, env any) (any, error) {
return vm.Run(program, env)
}
@@ -0,0 +1,17 @@
package expr
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestMatchHost(t *testing.T) {
v, err := Eval(`
let url = "rtsp://user:pass@192.168.1.123/cam/realmonitor?...";
let host = match(url, "//[^/]+")[0][2:];
host
`, nil)
require.Nil(t, err)
require.Equal(t, "user:pass@192.168.1.123", v)
}
@@ -0,0 +1,68 @@
## FFplay output
[FFplay](https://stackoverflow.com/questions/27778678/what-are-mv-fd-aq-vq-sq-and-f-in-a-video-stream) `7.11 A-V: 0.003 fd= 1 aq= 21KB vq= 321KB sq= 0B f=0/0`:
- `7.11` - master clock, is the time from start of the stream/video
- `A-V` - av_diff, difference between audio and video timestamps
- `fd` - frames dropped
- `aq` - audio queue (0 - no delay)
- `vq` - video queue (0 - no delay)
- `sq` - subtitle queue
- `f` - timestamp error correction rate (Not 100% sure)
`M-V`, `M-A` means video stream only, audio stream only respectively.
## Devices Windows
```
>ffmpeg -hide_banner -f dshow -list_options true -i video="VMware Virtual USB Video Device"
[dshow @ 0000025695e52900] DirectShow video device options (from video devices)
[dshow @ 0000025695e52900] Pin "Record" (alternative pin name "0")
[dshow @ 0000025695e52900] pixel_format=yuyv422 min s=1280x720 fps=1 max s=1280x720 fps=10
[dshow @ 0000025695e52900] pixel_format=yuyv422 min s=1280x720 fps=1 max s=1280x720 fps=10 (tv, bt470bg/bt709/unknown, topleft)
[dshow @ 0000025695e52900] pixel_format=nv12 min s=1280x720 fps=1 max s=1280x720 fps=23
[dshow @ 0000025695e52900] pixel_format=nv12 min s=1280x720 fps=1 max s=1280x720 fps=23 (tv, bt470bg/bt709/unknown, topleft)
```
## Devices Mac
```
% ./ffmpeg -hide_banner -f avfoundation -list_devices true -i ""
[AVFoundation indev @ 0x7f8b1f504d80] AVFoundation video devices:
[AVFoundation indev @ 0x7f8b1f504d80] [0] FaceTime HD Camera
[AVFoundation indev @ 0x7f8b1f504d80] [1] Capture screen 0
[AVFoundation indev @ 0x7f8b1f504d80] AVFoundation audio devices:
[AVFoundation indev @ 0x7f8b1f504d80] [0] Soundflower (2ch)
[AVFoundation indev @ 0x7f8b1f504d80] [1] Built-in Microphone
[AVFoundation indev @ 0x7f8b1f504d80] [2] Soundflower (64ch)
```
## Devices Linux
```
# ffmpeg -hide_banner -f v4l2 -list_formats all -i /dev/video0
[video4linux2,v4l2 @ 0x7f7de7c58bc0] Raw : yuyv422 : YUYV 4:2:2 : 640x480 160x120 176x144 320x176 320x240 352x288 432x240 544x288 640x360 752x416 800x448 800x600 864x480 960x544 960x720 1024x576 1184x656 1280x720 1280x960
[video4linux2,v4l2 @ 0x7f7de7c58bc0] Compressed: mjpeg : Motion-JPEG : 640x480 160x120 176x144 320x176 320x240 352x288 432x240 544x288 640x360 752x416 800x448 800x600 864x480 960x544 960x720 1024x576 1184x656 1280x720 1280x960
```
## TTS
```yaml
streams:
tts: ffmpeg:#input=-readrate 1 -readrate_initial_burst 0.001 -f lavfi -i "flite=text='1 2 3 4 5 6 7 8 9 0'"#audio=pcma
```
## Useful links
- https://superuser.com/questions/564402/explanation-of-x264-tune
- https://stackoverflow.com/questions/33624016/why-sliced-thread-affect-so-much-on-realtime-encoding-using-ffmpeg-x264
- https://codec.fandom.com/ru/wiki/X264_-_описание_ключей_кодирования
- https://html5test.com/
- https://trac.ffmpeg.org/wiki/Capture/Webcam
- https://trac.ffmpeg.org/wiki/DirectShow
- https://stackoverflow.com/questions/53207692/libav-mjpeg-encoding-and-huffman-table
- https://github.com/tuupola/esp_video/blob/master/README.md
- https://github.com/leandromoreira/ffmpeg-libav-tutorial
- https://www.reddit.com/user/VeritablePornocopium/comments/okw130/ffmpeg_with_libfdk_aac_for_windows_x64/
- https://slhck.info/video/2017/02/24/vbr-settings.html
- [HomeKit audio samples problem](https://superuser.com/questions/1290996/non-monotonous-dts-with-igndts-flag)
@@ -0,0 +1,123 @@
package ffmpeg
import (
"bytes"
"strconv"
"strings"
)
// correlation of libavformat versions with ffmpeg versions
const (
Version50 = "59. 16"
Version51 = "59. 27"
Version60 = "60. 3"
Version61 = "60. 16"
Version70 = "61. 1"
)
type Args struct {
Bin string // ffmpeg
Global string // -hide_banner -v error
Input string // -re -stream_loop -1 -i /media/bunny.mp4
Codecs []string // -c:v libx264 -g:v 30 -preset:v ultrafast -tune:v zerolatency
Filters []string // scale=1920:1080
Output string // -f rtsp {output}
Version string // libavformat version, it's more reliable than the ffmpeg version
Video, Audio int // count of Video and Audio params
}
func (a *Args) AddCodec(codec string) {
a.Codecs = append(a.Codecs, codec)
}
func (a *Args) AddFilter(filter string) {
a.Filters = append(a.Filters, filter)
}
func (a *Args) InsertFilter(filter string) {
a.Filters = append([]string{filter}, a.Filters...)
}
func (a *Args) HasFilters(filters ...string) bool {
for _, f1 := range a.Filters {
for _, f2 := range filters {
if strings.HasPrefix(f1, f2) {
return true
}
}
}
return false
}
func (a *Args) String() string {
b := bytes.NewBuffer(make([]byte, 0, 512))
b.WriteString(a.Bin)
if a.Global != "" {
b.WriteByte(' ')
b.WriteString(a.Global)
}
b.WriteByte(' ')
// starting from FFmpeg 6.1 readrate=1 has default initial bust 0.5 sec
// it might make us miss the first couple seconds of the file
if strings.HasPrefix(a.Input, "-re ") && a.Version >= Version61 {
b.WriteString("-readrate_initial_burst 0.001 ")
}
b.WriteString(a.Input)
multimode := a.Video > 1 || a.Audio > 1
var iv, ia int
for _, codec := range a.Codecs {
// support multiple video and/or audio codecs
if multimode && len(codec) >= 5 {
switch codec[:5] {
case "-c:v ":
codec = "-map 0:v:0? " + strings.ReplaceAll(codec, ":v ", ":v:"+strconv.Itoa(iv)+" ")
iv++
case "-c:a ":
codec = "-map 0:a:0? " + strings.ReplaceAll(codec, ":a ", ":a:"+strconv.Itoa(ia)+" ")
ia++
}
}
b.WriteByte(' ')
b.WriteString(codec)
}
if len(a.Filters) > 0 {
for i, filter := range a.Filters {
if i == 0 {
b.WriteString(` -vf "`)
} else {
b.WriteByte(',')
}
b.WriteString(filter)
}
b.WriteByte('"')
}
b.WriteByte(' ')
b.WriteString(a.Output)
return b.String()
}
func ParseVersion(b []byte) (ffmpeg string, libavformat string) {
if len(b) > 100 {
// ffmpeg version n7.0-30-g8b0fe91754-20240520 Copyright (c) 2000-2024 the FFmpeg developers
if i := bytes.IndexByte(b[15:], ' '); i > 0 {
ffmpeg = string(b[15 : 15+i])
}
// libavformat 60. 16.100 / 60. 16.100
if i := strings.Index(string(b), "libavformat"); i > 0 {
libavformat = string(b[i+15 : i+25])
}
}
return
}
@@ -0,0 +1,176 @@
package flussonic
import (
"strings"
"github.com/AlexxIT/go2rtc/pkg/aac"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/h264"
"github.com/AlexxIT/go2rtc/pkg/iso"
"github.com/gorilla/websocket"
"github.com/pion/rtp"
)
type Producer struct {
core.Connection
conn *websocket.Conn
videoTrackID, audioTrackID uint32
videoTimeScale, audioTimeScale float32
}
func Dial(source string) (core.Producer, error) {
url, _ := strings.CutPrefix(source, "flussonic:")
conn, _, err := websocket.DefaultDialer.Dial(url, nil)
if err != nil {
return nil, err
}
prod := &Producer{
Connection: core.Connection{
ID: core.NewID(),
FormatName: "flussonic",
Protocol: core.Before(url, ":"), // wss
RemoteAddr: conn.RemoteAddr().String(),
URL: url,
Transport: conn,
},
conn: conn,
}
if err = prod.probe(); err != nil {
_ = conn.Close()
return nil, err
}
return prod, nil
}
func (p *Producer) probe() error {
var init struct {
//Metadata struct {
// Tracks []struct {
// Width int `json:"width,omitempty"`
// Height int `json:"height,omitempty"`
// Fps int `json:"fps,omitempty"`
// Content string `json:"content"`
// TrackId string `json:"trackId"`
// Bitrate int `json:"bitrate"`
// } `json:"tracks"`
//} `json:"metadata"`
Tracks []struct {
Content string `json:"content"`
Id uint32 `json:"id"`
Payload []byte `json:"payload"`
} `json:"tracks"`
//Type string `json:"type"`
}
if err := p.conn.ReadJSON(&init); err != nil {
return err
}
var timeScale uint32
for _, track := range init.Tracks {
atoms, _ := iso.DecodeAtoms(track.Payload)
for _, atom := range atoms {
switch atom := atom.(type) {
case *iso.AtomMdhd:
timeScale = atom.TimeScale
case *iso.AtomVideo:
switch atom.Name {
case "avc1":
codec := h264.AVCCToCodec(atom.Config)
p.Medias = append(p.Medias, &core.Media{
Kind: core.KindVideo,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{codec},
})
p.videoTrackID = track.Id
p.videoTimeScale = float32(codec.ClockRate) / float32(timeScale)
}
case *iso.AtomAudio:
switch atom.Name {
case "mp4a":
codec := aac.ConfigToCodec(atom.Config)
p.Medias = append(p.Medias, &core.Media{
Kind: core.KindAudio,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{codec},
})
p.audioTrackID = track.Id
p.audioTimeScale = float32(codec.ClockRate) / float32(timeScale)
}
}
}
}
return nil
}
func (p *Producer) Start() error {
if err := p.conn.WriteMessage(websocket.TextMessage, []byte("resume")); err != nil {
return err
}
receivers := make(map[uint32]*core.Receiver)
timeScales := make(map[uint32]float32)
for _, receiver := range p.Receivers {
switch receiver.Codec.Kind() {
case core.KindVideo:
receivers[p.videoTrackID] = receiver
timeScales[p.videoTrackID] = p.videoTimeScale
case core.KindAudio:
receivers[p.audioTrackID] = receiver
timeScales[p.audioTrackID] = p.audioTimeScale
}
}
ch := make(chan []byte, 10)
defer close(ch)
go func() {
for b := range ch {
atoms, err := iso.DecodeAtoms(b)
if err != nil {
continue
}
var trackID uint32
var decodeTime uint64
for _, atom := range atoms {
switch atom := atom.(type) {
case *iso.AtomTfhd:
trackID = atom.TrackID
case *iso.AtomTfdt:
decodeTime = atom.DecodeTime
case *iso.AtomMdat:
b = atom.Data
}
}
if recv := receivers[trackID]; recv != nil {
timestamp := uint32(float32(decodeTime) * timeScales[trackID])
packet := &rtp.Packet{
Header: rtp.Header{Timestamp: timestamp},
Payload: b,
}
recv.WriteRTP(packet)
}
}
}()
for {
mType, b, err := p.conn.ReadMessage()
if err != nil {
return err
}
if mType == websocket.BinaryMessage {
p.Recv += len(b)
ch <- b
}
}
}
+239
View File
@@ -0,0 +1,239 @@
package amf
import (
"encoding/binary"
"errors"
"math"
)
const (
TypeNumber byte = iota
TypeBoolean
TypeString
TypeObject
TypeNull = 5
TypeEcmaArray = 8
TypeObjectEnd = 9
)
// AMF spec: http://download.macromedia.com/pub/labs/amf/amf0_spec_121207.pdf
type AMF struct {
buf []byte
pos int
}
var ErrRead = errors.New("amf: read error")
func NewReader(b []byte) *AMF {
return &AMF{buf: b}
}
func (a *AMF) ReadItems() ([]any, error) {
var items []any
for a.pos < len(a.buf) {
v, err := a.ReadItem()
if err != nil {
return nil, err
}
items = append(items, v)
}
return items, nil
}
func (a *AMF) ReadItem() (any, error) {
dataType, err := a.ReadByte()
if err != nil {
return nil, err
}
switch dataType {
case TypeNumber:
return a.ReadNumber()
case TypeBoolean:
b, err := a.ReadByte()
return b != 0, err
case TypeString:
return a.ReadString()
case TypeObject:
return a.ReadObject()
case TypeEcmaArray:
return a.ReadEcmaArray()
case TypeNull:
return nil, nil
case TypeObjectEnd:
return nil, nil
}
return nil, ErrRead
}
func (a *AMF) ReadByte() (byte, error) {
if a.pos >= len(a.buf) {
return 0, ErrRead
}
v := a.buf[a.pos]
a.pos++
return v, nil
}
func (a *AMF) ReadNumber() (float64, error) {
if a.pos+8 > len(a.buf) {
return 0, ErrRead
}
v := binary.BigEndian.Uint64(a.buf[a.pos : a.pos+8])
a.pos += 8
return math.Float64frombits(v), nil
}
func (a *AMF) ReadString() (string, error) {
if a.pos+2 > len(a.buf) {
return "", ErrRead
}
size := int(binary.BigEndian.Uint16(a.buf[a.pos:]))
a.pos += 2
if a.pos+size > len(a.buf) {
return "", ErrRead
}
s := string(a.buf[a.pos : a.pos+size])
a.pos += size
return s, nil
}
func (a *AMF) ReadObject() (map[string]any, error) {
obj := make(map[string]any)
for {
k, err := a.ReadString()
if err != nil {
return nil, err
}
v, err := a.ReadItem()
if err != nil {
return nil, err
}
if k == "" {
break
}
obj[k] = v
}
return obj, nil
}
func (a *AMF) ReadEcmaArray() (map[string]any, error) {
if a.pos+4 > len(a.buf) {
return nil, ErrRead
}
a.pos += 4 // skip size
return a.ReadObject()
}
func NewWriter() *AMF {
return &AMF{}
}
func (a *AMF) Bytes() []byte {
return a.buf
}
func (a *AMF) WriteNumber(n float64) {
b := math.Float64bits(n)
a.buf = append(
a.buf, TypeNumber,
byte(b>>56), byte(b>>48), byte(b>>40), byte(b>>32),
byte(b>>24), byte(b>>16), byte(b>>8), byte(b),
)
}
func (a *AMF) WriteBool(b bool) {
if b {
a.buf = append(a.buf, TypeBoolean, 1)
} else {
a.buf = append(a.buf, TypeBoolean, 0)
}
}
func (a *AMF) WriteString(s string) {
n := len(s)
a.buf = append(a.buf, TypeString, byte(n>>8), byte(n))
a.buf = append(a.buf, s...)
}
func (a *AMF) WriteObject(obj map[string]any) {
a.buf = append(a.buf, TypeObject)
a.writeKV(obj)
a.buf = append(a.buf, 0, 0, TypeObjectEnd)
}
func (a *AMF) WriteEcmaArray(obj map[string]any) {
n := len(obj)
a.buf = append(a.buf, TypeEcmaArray, byte(n>>24), byte(n>>16), byte(n>>8), byte(n))
a.writeKV(obj)
a.buf = append(a.buf, 0, 0, TypeObjectEnd)
}
func (a *AMF) writeKV(obj map[string]any) {
for k, v := range obj {
n := len(k)
a.buf = append(a.buf, byte(n>>8), byte(n))
a.buf = append(a.buf, k...)
switch v := v.(type) {
case string:
a.WriteString(v)
case int:
a.WriteNumber(float64(v))
case uint16:
a.WriteNumber(float64(v))
case uint32:
a.WriteNumber(float64(v))
case float64:
a.WriteNumber(v)
case bool:
a.WriteBool(v)
default:
panic(v)
}
}
}
func (a *AMF) WriteNull() {
a.buf = append(a.buf, TypeNull)
}
func EncodeItems(items ...any) []byte {
a := &AMF{}
for _, item := range items {
switch v := item.(type) {
case float64:
a.WriteNumber(v)
case int:
a.WriteNumber(float64(v))
case string:
a.WriteString(v)
case map[string]any:
a.WriteObject(v)
case nil:
a.WriteNull()
default:
panic(v)
}
}
return a.Bytes()
}
@@ -0,0 +1,281 @@
package amf
import (
"encoding/hex"
"testing"
"github.com/stretchr/testify/require"
)
func TestNewReader(t *testing.T) {
tests := []struct {
name string
actual string
expect []any
}{
{
name: "ffmpeg-http",
actual: "02000a6f6e4d65746144617461080000001000086475726174696f6e000000000000000000000577696474680040940000000000000006686569676874004086800000000000000d766964656f646174617261746500409e62770000000000096672616d6572617465004038000000000000000c766964656f636f646563696400401c000000000000000d617564696f646174617261746500405ea93000000000000f617564696f73616d706c65726174650040e5888000000000000f617564696f73616d706c6573697a65004030000000000000000673746572656f0101000c617564696f636f6465636964004024000000000000000b6d616a6f725f6272616e640200046d703432000d6d696e6f725f76657273696f6e020001300011636f6d70617469626c655f6272616e647302000c69736f6d617663316d7034320007656e636f64657202000c4c61766636302e352e313030000866696c6573697a65000000000000000000000009",
expect: []any{
"onMetaData",
map[string]any{
"compatible_brands": "isomavc1mp42",
"major_brand": "mp42",
"minor_version": "0",
"encoder": "Lavf60.5.100",
"filesize": float64(0),
"duration": float64(0),
"videocodecid": float64(7),
"width": float64(1280),
"height": float64(720),
"framerate": float64(24),
"videodatarate": 1944.6162109375,
"audiocodecid": float64(10),
"audiosamplerate": float64(44100),
"stereo": true,
"audiosamplesize": float64(16),
"audiodatarate": 122.6435546875,
},
},
},
{
name: "ffmpeg-file",
actual: "02000a6f6e4d65746144617461080000000800086475726174696f6e004000000000000000000577696474680040940000000000000006686569676874004086800000000000000d766964656f646174617261746500000000000000000000096672616d6572617465004039000000000000000c766964656f636f646563696400401c0000000000000007656e636f64657202000c4c61766636302e352e313030000866696c6573697a6500411f541400000000000009",
expect: []any{
"onMetaData",
map[string]any{
"encoder": "Lavf60.5.100",
"filesize": float64(513285),
"duration": float64(2),
"videocodecid": float64(7),
"width": float64(1280),
"height": float64(720),
"framerate": float64(25),
"videodatarate": float64(0),
},
},
},
{
name: "reolink-1",
actual: "0200075f726573756c74003ff0000000000000030006666d7356657202000d464d532f332c302c312c313233000c6361706162696c697469657300403f0000000000000000090300056c6576656c0200067374617475730004636f646502001d4e6574436f6e6e656374696f6e2e436f6e6e6563742e53756363657373000b6465736372697074696f6e020015436f6e6e656374696f6e207375636365656465642e000e6f626a656374456e636f64696e67000000000000000000000009",
expect: []any{
"_result", float64(1),
map[string]any{
"capabilities": float64(31),
"fmsVer": "FMS/3,0,1,123",
},
map[string]any{
"code": "NetConnection.Connect.Success",
"description": "Connection succeeded.",
"level": "status",
"objectEncoding": float64(0),
},
},
},
{
name: "reolink-2",
actual: "0200075f726573756c7400400000000000000005003ff0000000000000",
expect: []any{
"_result", float64(2), nil, float64(1),
},
},
{
name: "reolink-3",
actual: "0200086f6e537461747573000000000000000000050300056c6576656c0200067374617475730004636f64650200144e657453747265616d2e506c61792e5374617274000b6465736372697074696f6e020015537461727420766964656f206f6e2064656d616e64000009",
expect: []any{
"onStatus", float64(0), nil,
map[string]any{
"code": "NetStream.Play.Start",
"description": "Start video on demand",
"level": "status",
},
},
},
{
name: "reolink-4",
actual: "0200117c52746d7053616d706c6541636365737301010101",
expect: []any{
"|RtmpSampleAccess", true, true,
},
},
{
name: "reolink-5",
actual: "02000a6f6e4d6574614461746103000577696474680040a4000000000000000668656967687400409e000000000000000c646973706c617957696474680040a4000000000000000d646973706c617948656967687400409e00000000000000086475726174696f6e000000000000000000000c766964656f636f646563696400401c000000000000000c617564696f636f6465636964004024000000000000000f617564696f73616d706c65726174650040cf40000000000000096672616d657261746500403e000000000000000009",
expect: []any{
"onMetaData",
map[string]any{
"duration": float64(0),
"videocodecid": float64(7),
"width": float64(2560),
"height": float64(1920),
"displayWidth": float64(2560),
"displayHeight": float64(1920),
"framerate": float64(30),
"audiocodecid": float64(10),
"audiosamplerate": float64(16000),
},
},
},
{
name: "mediamtx",
actual: "02000d40736574446174614672616d6502000a6f6e4d6574614461746103000d766964656f6461746172617465000000000000000000000c766964656f636f646563696400401c000000000000000d617564696f6461746172617465000000000000000000000c617564696f636f6465636964004024000000000000000009",
expect: []any{
"@setDataFrame",
"onMetaData",
map[string]any{
"videocodecid": float64(7),
"videodatarate": float64(0),
"audiocodecid": float64(10),
"audiodatarate": float64(0),
},
},
},
{
name: "mediamtx",
actual: "0200075f726573756c74003ff0000000000000030006666d7356657202000d4c4e5820392c302c3132342c32000c6361706162696c697469657300403f0000000000000000090300056c6576656c0200067374617475730004636f646502001d4e6574436f6e6e656374696f6e2e436f6e6e6563742e53756363657373000b6465736372697074696f6e020015436f6e6e656374696f6e207375636365656465642e000e6f626a656374456e636f64696e67000000000000000000000009",
expect: []any{
"_result", float64(1), map[string]any{
"capabilities": float64(31),
"fmsVer": "LNX 9,0,124,2",
}, map[string]any{
"code": "NetConnection.Connect.Success",
"description": "Connection succeeded.",
"level": "status",
"objectEncoding": float64(0),
},
},
},
{
name: "mediamtx",
actual: "0200075f726573756c7400401000000000000005003ff0000000000000",
expect: []any{"_result", float64(4), any(nil), float64(1)},
},
{
name: "mediamtx",
actual: "0200086f6e537461747573004014000000000000050300056c6576656c0200067374617475730004636f64650200144e657453747265616d2e506c61792e5265736574000b6465736372697074696f6e02000a706c6179207265736574000009",
expect: []any{
"onStatus", float64(5), any(nil), map[string]any{
"code": "NetStream.Play.Reset",
"description": "play reset",
"level": "status",
},
},
},
{
name: "mediamtx",
actual: "0200086f6e537461747573004014000000000000050300056c6576656c0200067374617475730004636f64650200144e657453747265616d2e506c61792e5374617274000b6465736372697074696f6e02000a706c6179207374617274000009",
expect: []any{
"onStatus", float64(5), any(nil), map[string]any{
"code": "NetStream.Play.Start",
"description": "play start",
"level": "status",
},
},
},
{
name: "mediamtx",
actual: "0200086f6e537461747573004014000000000000050300056c6576656c0200067374617475730004636f64650200144e657453747265616d2e446174612e5374617274000b6465736372697074696f6e02000a64617461207374617274000009",
expect: []any{
"onStatus", float64(5), any(nil), map[string]any{
"code": "NetStream.Data.Start",
"description": "data start",
"level": "status",
},
},
},
{
name: "mediamtx",
actual: "0200086f6e537461747573004014000000000000050300056c6576656c0200067374617475730004636f646502001c4e657453747265616d2e506c61792e5075626c6973684e6f74696679000b6465736372697074696f6e02000e7075626c697368206e6f74696679000009",
expect: []any{
"onStatus", float64(5), any(nil), map[string]any{
"code": "NetStream.Play.PublishNotify",
"description": "publish notify",
"level": "status",
},
},
},
{
name: "obs-connect",
actual: "020007636f6e6e656374003ff000000000000003000361707002000c617070312f73747265616d3100047479706502000a6e6f6e70726976617465000e737570706f727473476f4177617901010008666c61736856657202001f464d4c452f332e302028636f6d70617469626c653b20464d53632f312e3029000673776655726c02002272746d703a2f2f3139322e3136382e31302e3130312f617070312f73747265616d310005746355726c02002272746d703a2f2f3139322e3136382e31302e3130312f617070312f73747265616d31000009",
expect: []any{
"connect", float64(1),
map[string]any{
"app": "app1/stream1",
"flashVer": "FMLE/3.0 (compatible; FMSc/1.0)",
"supportsGoAway": true,
"swfUrl": "rtmp://192.168.10.101/app1/stream1",
"tcUrl": "rtmp://192.168.10.101/app1/stream1",
"type": "nonprivate",
},
},
},
{
name: "obs-key",
actual: "02000d72656c6561736553747265616d004000000000000000050200046b657931",
expect: []any{
"releaseStream", float64(2), nil, "key1",
},
},
{
name: "obs",
actual: "02000d40736574446174614672616d6502000a6f6e4d65746144617461080000001400086475726174696f6e000000000000000000000866696c6553697a65000000000000000000000577696474680040840000000000000006686569676874004076800000000000000c766964656f636f646563696400401c000000000000000d766964656f64617461726174650040a388000000000000096672616d6572617465004039000000000000000c617564696f636f6465636964004024000000000000000d617564696f6461746172617465004064000000000000000f617564696f73616d706c65726174650040e5888000000000000f617564696f73616d706c6573697a65004030000000000000000d617564696f6368616e6e656c73004000000000000000000673746572656f01010003322e3101000003332e3101000003342e3001000003342e3101000003352e3101000003372e3101000007656e636f6465720200376f62732d6f7574707574206d6f64756c6520286c69626f62732076657273696f6e2032392e302e302d36322d6739303031323131663829000009",
expect: []any{
"@setDataFrame", "onMetaData", map[string]any{
"2.1": false,
"3.1": false,
"4.0": false,
"4.1": false,
"5.1": false,
"7.1": false,
"audiochannels": float64(2),
"audiocodecid": float64(10),
"audiodatarate": float64(160),
"audiosamplerate": float64(44100),
"audiosamplesize": float64(16),
"duration": float64(0),
"encoder": "obs-output module (libobs version 29.0.0-62-g9001211f8)",
"fileSize": float64(0),
"framerate": float64(25),
"height": float64(360),
"stereo": true,
"videocodecid": float64(7),
"videodatarate": float64(2500),
"width": float64(640),
},
},
},
{
name: "telegram-2",
actual: "0200075f726573756c7400400000000000000005",
expect: []any{
"_result", float64(2), nil,
},
},
{
name: "telegram-4",
actual: "0200075f726573756c7400401000000000000005003ff0000000000000",
expect: []any{
"_result", float64(4), nil, float64(1),
},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
b, err := hex.DecodeString(test.actual)
require.Nil(t, err)
rd := NewReader(b)
v, err := rd.ReadItems()
require.Nil(t, err)
require.Equal(t, test.expect, v)
})
}
}
@@ -0,0 +1,94 @@
package flv
import (
"io"
"github.com/AlexxIT/go2rtc/pkg/aac"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/h264"
"github.com/pion/rtp"
)
type Consumer struct {
core.Connection
wr *core.WriteBuffer
muxer *Muxer
}
func NewConsumer() *Consumer {
medias := []*core.Media{
{
Kind: core.KindVideo,
Direction: core.DirectionSendonly,
Codecs: []*core.Codec{
{Name: core.CodecH264},
},
},
{
Kind: core.KindAudio,
Direction: core.DirectionSendonly,
Codecs: []*core.Codec{
{Name: core.CodecAAC},
},
},
}
wr := core.NewWriteBuffer(nil)
return &Consumer{
Connection: core.Connection{
ID: core.NewID(),
FormatName: "flv",
Medias: medias,
Transport: wr,
},
wr: wr,
muxer: &Muxer{},
}
}
func (c *Consumer) AddTrack(media *core.Media, codec *core.Codec, track *core.Receiver) error {
sender := core.NewSender(media, track.Codec)
switch track.Codec.Name {
case core.CodecH264:
payload := c.muxer.GetPayloader(track.Codec)
sender.Handler = func(pkt *rtp.Packet) {
b := payload(pkt)
if n, err := c.wr.Write(b); err == nil {
c.Send += n
}
}
if track.Codec.IsRTP() {
sender.Handler = h264.RTPDepay(track.Codec, sender.Handler)
} else {
sender.Handler = h264.RepairAVCC(track.Codec, sender.Handler)
}
case core.CodecAAC:
payload := c.muxer.GetPayloader(track.Codec)
sender.Handler = func(pkt *rtp.Packet) {
b := payload(pkt)
if n, err := c.wr.Write(b); err == nil {
c.Send += n
}
}
if track.Codec.IsRTP() {
sender.Handler = aac.RTPDepay(sender.Handler)
}
}
sender.HandleRTP(track)
c.Senders = append(c.Senders, sender)
return nil
}
func (c *Consumer) WriteTo(wr io.Writer) (int64, error) {
b := c.muxer.GetInit()
if _, err := wr.Write(b); err != nil {
return 0, err
}
return c.wr.WriteTo(wr)
}
@@ -0,0 +1,21 @@
package flv
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestTimeToRTP(t *testing.T) {
// Reolink camera has 20 FPS
// Video timestamp increases by 50ms, SampleRate 90000, RTP timestamp increases by 4500
// Audio timestamp increases by 64ms, SampleRate 16000, RTP timestamp increases by 1024
frameN := 1
for i := 0; i < 32; i++ {
// 1000ms/(90000/4500) = 50ms
require.Equal(t, uint32(frameN*4500), TimeToRTP(uint32(frameN*50), 90000))
// 1000ms/(16000/1024) = 64ms
require.Equal(t, uint32(frameN*1024), TimeToRTP(uint32(frameN*64), 16000))
frameN *= 2
}
}
+174
View File
@@ -0,0 +1,174 @@
package flv
import (
"encoding/binary"
"encoding/hex"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/flv/amf"
"github.com/AlexxIT/go2rtc/pkg/h264"
"github.com/pion/rtp"
)
type Muxer struct {
codecs []*core.Codec
}
const (
FlagsVideo = 0b001
FlagsAudio = 0b100
)
func (m *Muxer) GetInit() []byte {
b := []byte{
'F', 'L', 'V', // signature
1, // version
0, // flags (has video/audio)
0, 0, 0, 9, // header size
0, 0, 0, 0, // tag 0 size
}
obj := map[string]any{}
for _, codec := range m.codecs {
switch codec.Name {
case core.CodecH264:
b[4] |= FlagsVideo
obj["videocodecid"] = CodecH264
case core.CodecAAC:
b[4] |= FlagsAudio
obj["audiocodecid"] = CodecAAC
obj["audiosamplerate"] = codec.ClockRate
obj["audiosamplesize"] = 16
obj["stereo"] = codec.Channels == 2
}
}
data := amf.EncodeItems("@setDataFrame", "onMetaData", obj)
b = append(b, EncodeTag(TagData, 0, data)...)
for _, codec := range m.codecs {
switch codec.Name {
case core.CodecH264:
sps, pps := h264.GetParameterSet(codec.FmtpLine)
if len(sps) == 0 {
sps = []byte{0x67, 0x42, 0x00, 0x0a, 0xf8, 0x41, 0xa2}
} else {
h264.FixPixFmt(sps)
}
if len(pps) == 0 {
pps = []byte{0x68, 0xce, 0x38, 0x80}
}
config := h264.EncodeConfig(sps, pps)
video := append(encodeAVData(codec, 0), config...)
b = append(b, EncodeTag(TagVideo, 0, video)...)
case core.CodecAAC:
s := core.Between(codec.FmtpLine, "config=", ";")
config, _ := hex.DecodeString(s)
audio := append(encodeAVData(codec, 0), config...)
b = append(b, EncodeTag(TagAudio, 0, audio)...)
}
}
return b
}
func (m *Muxer) GetPayloader(codec *core.Codec) func(packet *rtp.Packet) []byte {
m.codecs = append(m.codecs, codec)
var ts0 uint32
var k = codec.ClockRate / 1000
switch codec.Name {
case core.CodecH264:
buf := encodeAVData(codec, 1)
return func(packet *rtp.Packet) []byte {
if h264.IsKeyframe(packet.Payload) {
buf[0] = 1<<4 | 7
} else {
buf[0] = 2<<4 | 7
}
buf = append(buf[:5], packet.Payload...) // reset buffer to previous place
if ts0 == 0 {
ts0 = packet.Timestamp
}
timeMS := (packet.Timestamp - ts0) / k
return EncodeTag(TagVideo, timeMS, buf)
}
case core.CodecAAC:
buf := encodeAVData(codec, 1)
return func(packet *rtp.Packet) []byte {
buf = append(buf[:2], packet.Payload...)
if ts0 == 0 {
ts0 = packet.Timestamp
}
timeMS := (packet.Timestamp - ts0) / k
return EncodeTag(TagAudio, timeMS, buf)
}
}
return nil
}
func EncodeTag(tagType byte, timeMS uint32, payload []byte) []byte {
payloadSize := uint32(len(payload))
tagSize := payloadSize + 11
b := make([]byte, tagSize+4)
b[0] = tagType
b[1] = byte(payloadSize >> 16)
b[2] = byte(payloadSize >> 8)
b[3] = byte(payloadSize)
b[4] = byte(timeMS >> 16)
b[5] = byte(timeMS >> 8)
b[6] = byte(timeMS)
b[7] = byte(timeMS >> 24)
copy(b[11:], payload)
binary.BigEndian.PutUint32(b[tagSize:], tagSize)
return b
}
func encodeAVData(codec *core.Codec, isFrame byte) []byte {
switch codec.Name {
case core.CodecH264:
return []byte{
1<<4 | 7, // keyframe + AVC
isFrame, // 0 - config, 1 - frame
0, 0, 0, // composition time = 0
}
case core.CodecAAC:
var b0 byte = 10 << 4 // AAC
switch codec.ClockRate {
case 11025:
b0 |= 1 << 2
case 22050:
b0 |= 2 << 2
case 44100:
b0 |= 3 << 2
}
b0 |= 1 << 1 // 16 bits
if codec.Channels == 2 {
b0 |= 1
}
return []byte{b0, isFrame} // 0 - config, 1 - frame
}
return nil
}
+312
View File
@@ -0,0 +1,312 @@
package flv
import (
"bytes"
"encoding/binary"
"errors"
"io"
"time"
"github.com/AlexxIT/go2rtc/pkg/aac"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/h264"
"github.com/AlexxIT/go2rtc/pkg/h265"
"github.com/pion/rtp"
)
type Producer struct {
core.Connection
rd *core.ReadBuffer
video, audio *core.Receiver
}
func Open(rd io.Reader) (*Producer, error) {
prod := &Producer{
Connection: core.Connection{
ID: core.NewID(),
FormatName: "flv",
Transport: rd,
},
rd: core.NewReadBuffer(rd),
}
if err := prod.probe(); err != nil {
return nil, err
}
return prod, nil
}
const (
Signature = "FLV"
TagAudio = 8
TagVideo = 9
TagData = 18
CodecAAC = 10
CodecH264 = 7
CodecHEVC = 12
)
const (
PacketTypeAVCHeader = iota
PacketTypeAVCNALU
PacketTypeAVCEnd
)
const (
PacketTypeSequenceStart = iota
PacketTypeCodedFrames
PacketTypeSequenceEnd
PacketTypeCodedFramesX
PacketTypeMetadata
PacketTypeMPEG2TSSequenceStart
)
func (c *Producer) GetTrack(media *core.Media, codec *core.Codec) (*core.Receiver, error) {
receiver, _ := c.Connection.GetTrack(media, codec)
if media.Kind == core.KindVideo {
c.video = receiver
} else {
c.audio = receiver
}
return receiver, nil
}
func (c *Producer) Start() error {
for {
pkt, err := c.readPacket()
if err != nil {
return err
}
c.Recv += len(pkt.Payload)
switch pkt.PayloadType {
case TagAudio:
if c.audio == nil || pkt.Payload[1] == 0 {
continue
}
pkt.Timestamp = TimeToRTP(pkt.Timestamp, c.audio.Codec.ClockRate)
pkt.Payload = pkt.Payload[2:]
c.audio.WriteRTP(pkt)
case TagVideo:
if c.video == nil {
continue
}
if isExHeader(pkt.Payload) {
switch packetType := pkt.Payload[0] & 0b1111; packetType {
case PacketTypeCodedFrames:
// frame type 4b, packet type 4b, fourCC 32b, composition time 24b
pkt.Payload = pkt.Payload[8:]
case PacketTypeCodedFramesX:
// frame type 4b, packet type 4b, fourCC 32b
pkt.Payload = pkt.Payload[5:]
default:
continue
}
} else {
switch pkt.Payload[1] {
case PacketTypeAVCNALU:
// frame type 4b, codecID 4b, avc packet type 8b, composition time 24b
pkt.Payload = pkt.Payload[5:]
default:
continue
}
}
pkt.Timestamp = TimeToRTP(pkt.Timestamp, c.video.Codec.ClockRate)
c.video.WriteRTP(pkt)
}
}
}
func (c *Producer) probe() error {
if err := c.readHeader(); err != nil {
return err
}
c.rd.BufferSize = core.ProbeSize
defer c.rd.Reset()
// Normal software sends:
// 1. Video/audio flag in header
// 2. MetaData as first tag (with video/audio codec info)
// 3. Video/audio headers in 2nd and 3rd tag
// Reolink camera sends:
// 1. Empty video/audio flag
// 2. MedaData without stereo key for AAC
// 3. Audio header after Video keyframe tag
// OpenIPC camera (on old firmwares) sends:
// 1. Empty video/audio flag
// 2. No MetaData packet
// 3. Sends a video packet in more than 3 seconds
waitVideo := true
waitAudio := true
timeout := time.Now().Add(time.Second * 5)
for (waitVideo || waitAudio) && time.Now().Before(timeout) {
pkt, err := c.readPacket()
if err != nil {
return err
}
//log.Printf("%d %0.20s", pkt.PayloadType, pkt.Payload)
switch pkt.PayloadType {
case TagAudio:
if !waitAudio {
continue
}
_ = pkt.Payload[1] // bounds
codecID := pkt.Payload[0] >> 4 // SoundFormat
_ = pkt.Payload[0] & 0b1100 // SoundRate
_ = pkt.Payload[0] & 0b0010 // SoundSize
_ = pkt.Payload[0] & 0b0001 // SoundType
if codecID != CodecAAC {
continue
}
if pkt.Payload[1] != 0 { // check if header
continue
}
codec := aac.ConfigToCodec(pkt.Payload[2:])
media := &core.Media{
Kind: core.KindAudio,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{codec},
}
c.Medias = append(c.Medias, media)
waitAudio = false
case TagVideo:
if !waitVideo {
continue
}
var codec *core.Codec
if isExHeader(pkt.Payload) {
if string(pkt.Payload[1:5]) != "hvc1" {
continue
}
if packetType := pkt.Payload[0] & 0b1111; packetType != PacketTypeSequenceStart {
continue
}
codec = h265.ConfigToCodec(pkt.Payload[5:])
} else {
_ = pkt.Payload[0] >> 4 // FrameType
if packetType := pkt.Payload[1]; packetType != PacketTypeAVCHeader { // check if header
continue
}
switch codecID := pkt.Payload[0] & 0b1111; codecID {
case CodecH264:
codec = h264.ConfigToCodec(pkt.Payload[5:])
case CodecHEVC:
codec = h265.ConfigToCodec(pkt.Payload[5:])
default:
continue
}
}
media := &core.Media{
Kind: core.KindVideo,
Direction: core.DirectionRecvonly,
Codecs: []*core.Codec{codec},
}
c.Medias = append(c.Medias, media)
waitVideo = false
case TagData:
if !bytes.Contains(pkt.Payload, []byte("onMetaData")) {
continue
}
// Dahua cameras doesn't send videocodecid
if !bytes.Contains(pkt.Payload, []byte("videocodecid")) &&
!bytes.Contains(pkt.Payload, []byte("width")) &&
!bytes.Contains(pkt.Payload, []byte("framerate")) {
waitVideo = false
}
if !bytes.Contains(pkt.Payload, []byte("audiocodecid")) {
waitAudio = false
}
}
}
return nil
}
func (c *Producer) readHeader() error {
b := make([]byte, 9)
if _, err := io.ReadFull(c.rd, b); err != nil {
return err
}
if string(b[:3]) != Signature {
return errors.New("flv: wrong header")
}
_ = b[4] // flags (skip because unsupported by Reolink cameras)
if skip := binary.BigEndian.Uint32(b[5:]) - 9; skip > 0 {
if _, err := io.ReadFull(c.rd, make([]byte, skip)); err != nil {
return err
}
}
return nil
}
func (c *Producer) readPacket() (*rtp.Packet, error) {
// https://rtmp.veriskope.com/pdf/video_file_format_spec_v10.pdf
b := make([]byte, 4+11)
if _, err := io.ReadFull(c.rd, b); err != nil {
return nil, err
}
b = b[4 : 4+11] // skip previous tag size
size := uint32(b[1])<<16 | uint32(b[2])<<8 | uint32(b[3])
pkt := &rtp.Packet{
Header: rtp.Header{
PayloadType: b[0],
Timestamp: uint32(b[4])<<16 | uint32(b[5])<<8 | uint32(b[6]) | uint32(b[7])<<24,
},
Payload: make([]byte, size),
}
if _, err := io.ReadFull(c.rd, pkt.Payload); err != nil {
return nil, err
}
//log.Printf("[FLV] %d %.40x", pkt.PayloadType, pkt.Payload)
return pkt, nil
}
// TimeToRTP convert time in milliseconds to RTP time
func TimeToRTP(timeMS, clockRate uint32) uint32 {
// for clockRates 90000, 16000, 8000, etc. - we can use:
// return timeMS * (clockRate / 1000)
// but for clockRates 44100, 22050, 11025 - we should use:
return uint32(uint64(timeMS) * uint64(clockRate) / 1000)
}
func isExHeader(data []byte) bool {
return data[0]&0b1000_0000 != 0
}
@@ -0,0 +1,43 @@
package gopro
import (
"net"
"net/http"
"regexp"
)
func Discovery() (urls []string) {
ints, err := net.Interfaces()
if err != nil {
return nil
}
// The socket address for USB connections is 172.2X.1YZ.51:8080
// https://gopro.github.io/OpenGoPro/http_2_0#socket-address
re := regexp.MustCompile(`^172\.2\d\.1\d\d\.`)
for _, itf := range ints {
addrs, err := itf.Addrs()
if err != nil {
continue
}
for _, addr := range addrs {
host := addr.String()
if !re.MatchString(host) {
continue
}
host = host[:11] + "51" // 172.2x.1xx.xxx
res, err := http.Get("http://" + host + ":8080/gopro/webcam/status")
if err != nil {
continue
}
_ = res.Body.Close()
urls = append(urls, host)
}
}
return
}
@@ -0,0 +1,124 @@
package gopro
import (
"errors"
"io"
"net"
"net/http"
"net/url"
"time"
"github.com/AlexxIT/go2rtc/pkg/mpegts"
)
func Dial(rawURL string) (*mpegts.Producer, error) {
u, err := url.Parse(rawURL)
if err != nil {
return nil, err
}
r := &listener{host: u.Host}
if err = r.command("/gopro/webcam/stop"); err != nil {
return nil, err
}
if err = r.listen(); err != nil {
return nil, err
}
if err = r.command("/gopro/webcam/start"); err != nil {
return nil, err
}
prod, err := mpegts.Open(r)
if err != nil {
return nil, err
}
prod.FormatName = "gopro"
prod.RemoteAddr = u.Host
return prod, nil
}
type listener struct {
conn net.PacketConn
host string
packet []byte
packets chan []byte
}
func (r *listener) Read(p []byte) (n int, err error) {
if r.packet == nil {
var ok bool
if r.packet, ok = <-r.packets; !ok {
return 0, io.EOF // channel closed
}
}
n = copy(p, r.packet)
if n < len(r.packet) {
r.packet = r.packet[n:]
} else {
r.packet = nil
}
return
}
func (r *listener) Close() error {
return r.conn.Close()
}
func (r *listener) command(api string) error {
client := &http.Client{Timeout: 5 * time.Second}
res, err := client.Get("http://" + r.host + ":8080" + api)
if err != nil {
return err
}
_ = res.Body.Close()
if res.StatusCode != http.StatusOK {
return errors.New("gopro: wrong response: " + res.Status)
}
return nil
}
func (r *listener) listen() (err error) {
if r.conn, err = net.ListenPacket("udp4", ":8554"); err != nil {
return
}
r.packets = make(chan []byte, 1024)
go r.worker()
return
}
func (r *listener) worker() {
b := make([]byte, 1500)
for {
if err := r.conn.SetReadDeadline(time.Now().Add(3 * time.Second)); err != nil {
break
}
n, _, err := r.conn.ReadFrom(b)
if err != nil {
break
}
packet := make([]byte, n)
copy(packet, b)
r.packets <- packet
}
close(r.packets)
_ = r.command("/gopro/webcam/stop")
}
@@ -0,0 +1,16 @@
# H264
Payloader code taken from [pion](https://github.com/pion/rtp) library and changed to AVC packets support.
## Useful Links
- [RTP Payload Format for H.264 Video](https://datatracker.ietf.org/doc/html/rfc6184)
- [The H264 Sequence parameter set](https://www.cardinalpeak.com/blog/the-h-264-sequence-parameter-set)
- [H.264 Video Types (Microsoft)](https://docs.microsoft.com/en-us/windows/win32/directshow/h-264-video-types)
- [Automatic Generation of H.264 Parameter Sets to Recover Video File Fragments](https://arxiv.org/pdf/2104.14522.pdf)
- [Chromium sources](https://chromium.googlesource.com/external/webrtc/+/HEAD/common_video/h264)
- [AVC levels](https://en.wikipedia.org/wiki/Advanced_Video_Coding#Levels)
- [AVC profiles table](https://developer.mozilla.org/ru/docs/Web/Media/Formats/codecs_parameter)
- [Supported Media for Google Cast](https://developers.google.com/cast/docs/media)
- [Two stream formats, Annex-B, AVCC (H.264) and HVCC (H.265)](https://www.programmersought.com/article/3901815022/)
- https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/producer-reference-nal.html
@@ -0,0 +1,156 @@
// Package annexb - universal for H264 and H265
package annexb
import (
"bytes"
"encoding/binary"
)
const StartCode = "\x00\x00\x00\x01"
const startAUD = StartCode + "\x09\xF0"
const startAUDstart = startAUD + StartCode
// EncodeToAVCC
//
// FFmpeg MPEG-TS: 00000001 AUD 00000001 SPS 00000001 PPS 000001 IFrame
// FFmpeg H264: 00000001 SPS 00000001 PPS 000001 IFrame 00000001 PFrame
// Reolink: 000001 AUD 000001 VPS 00000001 SPS 00000001 PPS 00000001 IDR 00000001 IDR
func EncodeToAVCC(annexb []byte) (avc []byte) {
var start int
avc = make([]byte, 0, len(annexb)+4) // init memory with little overhead
for i := 0; ; i++ {
var offset int
if i+3 < len(annexb) {
// search next separator
if annexb[i] == 0 && annexb[i+1] == 0 {
if annexb[i+2] == 1 {
offset = 3 // 00 00 01
} else if annexb[i+2] == 0 && annexb[i+3] == 1 {
offset = 4 // 00 00 00 01
} else {
continue
}
} else {
continue
}
} else {
i = len(annexb) // move i to data end
}
if start != 0 {
size := uint32(i - start)
avc = binary.BigEndian.AppendUint32(avc, size)
avc = append(avc, annexb[start:i]...)
}
// sometimes FFmpeg put separator at the end
if i += offset; i == len(annexb) {
break
}
if isAUD(annexb[i]) {
start = 0 // skip this NALU
} else {
start = i // save this position
}
}
return
}
func isAUD(b byte) bool {
const h264 = 9
const h265 = 35 << 1
return b&0b0001_1111 == h264 || b&0b0111_1110 == h265
}
func DecodeAVCC(b []byte, safeClone bool) []byte {
if safeClone {
b = bytes.Clone(b)
}
for i := 0; i < len(b); {
size := int(binary.BigEndian.Uint32(b[i:]))
b[i] = 0
b[i+1] = 0
b[i+2] = 0
b[i+3] = 1
i += 4 + size
}
return b
}
// DecodeAVCCWithAUD - AUD doesn't important for FFmpeg, but important for Safari
func DecodeAVCCWithAUD(src []byte) []byte {
dst := make([]byte, len(startAUD)+len(src))
copy(dst, startAUD)
copy(dst[len(startAUD):], src)
DecodeAVCC(dst[len(startAUD):], false)
return dst
}
const (
h264PFrame = 1
h264IFrame = 5
h264SPS = 7
h264PPS = 8
h265VPS = 32
h265PFrame = 1
)
// IndexFrame - get new frame start position in the AnnexB stream
func IndexFrame(b []byte) int {
if len(b) < len(startAUDstart) {
return -1
}
for i := len(startAUDstart); ; {
if di := bytes.Index(b[i:], []byte(StartCode)); di < 0 {
break
} else {
i += di + 4 // move to NALU start
}
if i >= len(b) {
break
}
h264Type := b[i] & 0b1_1111
switch h264Type {
case h264PFrame, h264SPS:
return i - 4 // move to start code
case h264IFrame, h264PPS:
continue
}
h265Type := (b[i] >> 1) & 0b11_1111
switch h265Type {
case h265PFrame, h265VPS:
return i - 4 // move to start code
}
}
return -1
}
func FixAnnexBInAVCC(b []byte) []byte {
for i := 0; i < len(b); {
if i+4 >= len(b) {
break
}
size := bytes.Index(b[i+4:], []byte{0, 0, 0, 1})
if size < 0 {
size = len(b) - (i + 4)
}
binary.BigEndian.PutUint32(b[i:], uint32(size))
i += size + 4
}
return b
}
File diff suppressed because one or more lines are too long
+122
View File
@@ -0,0 +1,122 @@
package h264
import (
"bytes"
"encoding/binary"
)
const forbiddenZeroBit = 0x80
const nalUnitType = 0x1F
// Deprecated: DecodeStream - find and return first AU in AVC format
// useful for processing live streams with unknown separator size
func DecodeStream(annexb []byte) ([]byte, int) {
startPos := -1
i := 0
for {
// search next separator
if i = IndexFrom(annexb, []byte{0, 0, 1}, i); i < 0 {
break
}
// move i to next AU
if i += 3; i >= len(annexb) {
break
}
// check if AU type valid
octet := annexb[i]
if octet&forbiddenZeroBit != 0 {
continue
}
// 0 => AUD => SPS/IF/PF => AUD
// 0 => SPS/PF => SPS/PF
nalType := octet & nalUnitType
if startPos >= 0 {
switch nalType {
case NALUTypeAUD, NALUTypeSPS, NALUTypePFrame:
if annexb[i-4] == 0 {
return DecodeAnnexB(annexb[startPos : i-4]), i - 4
} else {
return DecodeAnnexB(annexb[startPos : i-3]), i - 3
}
}
} else {
switch nalType {
case NALUTypeSPS, NALUTypePFrame:
if i >= 4 && annexb[i-4] == 0 {
startPos = i - 4
} else {
startPos = i - 3
}
}
}
}
return nil, 0
}
// DecodeAnnexB - convert AnnexB to AVC format
// support unknown separator size
func DecodeAnnexB(b []byte) []byte {
if b[2] == 1 {
// convert: 0 0 1 => 0 0 0 1
b = append([]byte{0}, b...)
}
startPos := 0
i := 4
for {
// search next separato
if i = IndexFrom(b, []byte{0, 0, 1}, i); i < 0 {
break
}
// move i to next AU
if i += 3; i >= len(b) {
break
}
// check if AU type valid
octet := b[i]
if octet&forbiddenZeroBit != 0 {
continue
}
switch octet & nalUnitType {
case NALUTypePFrame, NALUTypeIFrame, NALUTypeSPS, NALUTypePPS:
if b[i-4] != 0 {
// prefix: 0 0 1
binary.BigEndian.PutUint32(b[startPos:], uint32(i-startPos-7))
tmp := make([]byte, 0, len(b)+1)
tmp = append(tmp, b[:i]...)
tmp = append(tmp, 0)
b = append(tmp, b[i:]...)
startPos = i - 3
} else {
// prefix: 0 0 0 1
binary.BigEndian.PutUint32(b[startPos:], uint32(i-startPos-8))
startPos = i - 4
}
}
}
binary.BigEndian.PutUint32(b[startPos:], uint32(len(b)-startPos-4))
return b
}
func IndexFrom(b []byte, sep []byte, from int) int {
if from > 0 {
if from < len(b) {
if i := bytes.Index(b[from:], sep); i >= 0 {
return from + i
}
}
return -1
}
return bytes.Index(b, sep)
}
+120
View File
@@ -0,0 +1,120 @@
// Package h264 - AVCC format related functions
package h264
import (
"bytes"
"encoding/base64"
"encoding/binary"
"encoding/hex"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/pion/rtp"
)
func RepairAVCC(codec *core.Codec, handler core.HandlerFunc) core.HandlerFunc {
sps, pps := GetParameterSet(codec.FmtpLine)
ps := JoinNALU(sps, pps)
return func(packet *rtp.Packet) {
// this can happen for FLV from FFmpeg
if NALUType(packet.Payload) == NALUTypeSEI {
size := int(binary.BigEndian.Uint32(packet.Payload)) + 4
packet.Payload = packet.Payload[size:]
}
if NALUType(packet.Payload) == NALUTypeIFrame {
packet.Payload = Join(ps, packet.Payload)
}
handler(packet)
}
}
func JoinNALU(nalus ...[]byte) (avcc []byte) {
var i, n int
for _, nalu := range nalus {
if i = len(nalu); i > 0 {
n += 4 + i
}
}
avcc = make([]byte, n)
n = 0
for _, nal := range nalus {
if i = len(nal); i > 0 {
binary.BigEndian.PutUint32(avcc[n:], uint32(i))
n += 4 + copy(avcc[n+4:], nal)
}
}
return
}
func SplitNALU(avcc []byte) [][]byte {
var nals [][]byte
for {
// get AVC length
size := int(binary.BigEndian.Uint32(avcc)) + 4
// check if multiple items in one packet
if size < len(avcc) {
nals = append(nals, avcc[:size])
avcc = avcc[size:]
} else {
nals = append(nals, avcc)
break
}
}
return nals
}
func NALUTypes(avcc []byte) []byte {
var types []byte
for {
types = append(types, NALUType(avcc))
size := 4 + int(binary.BigEndian.Uint32(avcc))
if size < len(avcc) {
avcc = avcc[size:]
} else {
break
}
}
return types
}
func AVCCToCodec(avcc []byte) *core.Codec {
buf := bytes.NewBufferString("packetization-mode=1")
for {
n := len(avcc)
if n < 4 {
break
}
size := 4 + int(binary.BigEndian.Uint32(avcc))
if n < size {
break
}
switch NALUType(avcc) {
case NALUTypeSPS:
buf.WriteString(";profile-level-id=")
buf.WriteString(hex.EncodeToString(avcc[5:8]))
buf.WriteString(";sprop-parameter-sets=")
buf.WriteString(base64.StdEncoding.EncodeToString(avcc[4:size]))
case NALUTypePPS:
buf.WriteString(",")
buf.WriteString(base64.StdEncoding.EncodeToString(avcc[4:size]))
}
avcc = avcc[size:]
}
return &core.Codec{
Name: core.CodecH264,
ClockRate: 90000,
FmtpLine: buf.String(),
PayloadType: core.PayloadTypeRAW,
}
}
+145
View File
@@ -0,0 +1,145 @@
package h264
import (
"encoding/base64"
"encoding/binary"
"encoding/hex"
"fmt"
"strings"
"github.com/AlexxIT/go2rtc/pkg/core"
)
const (
NALUTypePFrame = 1 // Coded slice of a non-IDR picture
NALUTypeIFrame = 5 // Coded slice of an IDR picture
NALUTypeSEI = 6 // Supplemental enhancement information (SEI)
NALUTypeSPS = 7 // Sequence parameter set
NALUTypePPS = 8 // Picture parameter set
NALUTypeAUD = 9 // Access unit delimiter
)
func NALUType(b []byte) byte {
return b[4] & 0x1F
}
// IsKeyframe - check if any NALU in one AU is Keyframe
func IsKeyframe(b []byte) bool {
for {
switch NALUType(b) {
case NALUTypePFrame:
return false
case NALUTypeIFrame:
return true
}
size := int(binary.BigEndian.Uint32(b)) + 4
if size < len(b) {
b = b[size:]
continue
} else {
return false
}
}
}
func Join(ps, iframe []byte) []byte {
b := make([]byte, len(ps)+len(iframe))
i := copy(b, ps)
copy(b[i:], iframe)
return b
}
// https://developers.google.com/cast/docs/media
const (
ProfileBaseline = 0x42
ProfileMain = 0x4D
ProfileHigh = 0x64
CapabilityBaseline = 0xE0
CapabilityMain = 0x40
)
// GetProfileLevelID - get profile from fmtp line
// Some devices won't play video with high level, so limit max profile and max level.
// And return some profile even if fmtp line is empty.
func GetProfileLevelID(fmtp string) string {
// avc1.640029 - H.264 high 4.1 (Chromecast 1st and 2nd Gen)
profile := byte(ProfileHigh)
capab := byte(0)
level := byte(41)
if fmtp != "" {
var conf []byte
// some cameras has wrong profile-level-id
// https://github.com/AlexxIT/go2rtc/issues/155
if s := core.Between(fmtp, "sprop-parameter-sets=", ","); s != "" {
if sps, _ := base64.StdEncoding.DecodeString(s); len(sps) >= 4 {
conf = sps[1:4]
}
} else if s = core.Between(fmtp, "profile-level-id=", ";"); s != "" {
conf, _ = hex.DecodeString(s)
}
if len(conf) == 3 {
// sanitize profile, capab and level to supported values
switch conf[0] {
case ProfileBaseline, ProfileMain:
profile = conf[0]
}
switch conf[1] {
case CapabilityBaseline, CapabilityMain:
capab = conf[1]
}
switch conf[2] {
case 30, 31, 40:
level = conf[2]
}
}
}
return fmt.Sprintf("%02X%02X%02X", profile, capab, level)
}
func GetParameterSet(fmtp string) (sps, pps []byte) {
if fmtp == "" {
return
}
s := core.Between(fmtp, "sprop-parameter-sets=", ";")
if s == "" {
return
}
i := strings.IndexByte(s, ',')
if i < 0 {
return
}
sps, _ = base64.StdEncoding.DecodeString(s[:i])
pps, _ = base64.StdEncoding.DecodeString(s[i+1:])
return
}
// GetFmtpLine from SPS+PPS+IFrame in AVC format
func GetFmtpLine(avc []byte) string {
s := "packetization-mode=1"
for {
size := 4 + int(binary.BigEndian.Uint32(avc))
switch NALUType(avc) {
case NALUTypeSPS:
s += ";profile-level-id=" + hex.EncodeToString(avc[5:8])
s += ";sprop-parameter-sets=" + base64.StdEncoding.EncodeToString(avc[4:size])
case NALUTypePPS:
s += "," + base64.StdEncoding.EncodeToString(avc[4:size])
}
if size < len(avc) {
avc = avc[size:]
} else {
return s
}
}
}
@@ -0,0 +1,110 @@
package h264
import (
"encoding/base64"
"encoding/hex"
"testing"
"github.com/stretchr/testify/require"
)
func TestDecodeConfig(t *testing.T) {
s := "01640033ffe1000c67640033ac1514a02800f19001000468ee3cb0"
src, err := hex.DecodeString(s)
require.Nil(t, err)
profile, sps, pps := DecodeConfig(src)
require.NotNil(t, profile)
require.NotNil(t, sps)
require.NotNil(t, pps)
dst := EncodeConfig(sps, pps)
require.Equal(t, src, dst)
}
func TestDecodeSPS(t *testing.T) {
s := "Z0IAMukAUAHjQgAAB9IAAOqcCAA=" // Amcrest AD410
b, err := base64.StdEncoding.DecodeString(s)
require.Nil(t, err)
sps := DecodeSPS(b)
require.Equal(t, uint16(2560), sps.Width())
require.Equal(t, uint16(1920), sps.Height())
s = "R00AKZmgHgCJ+WEAAAMD6AAATiCE" // Sonoff
b, err = base64.StdEncoding.DecodeString(s)
require.Nil(t, err)
sps = DecodeSPS(b)
require.Equal(t, uint16(1920), sps.Width())
require.Equal(t, uint16(1080), sps.Height())
s = "Z01AMqaAKAC1kAA=" // Dahua
b, err = base64.StdEncoding.DecodeString(s)
require.Nil(t, err)
sps = DecodeSPS(b)
require.Equal(t, uint16(2560), sps.Width())
require.Equal(t, uint16(1440), sps.Height())
s = "Z2QAM6wVFKAoAPGQ" // Reolink
b, err = base64.StdEncoding.DecodeString(s)
require.Nil(t, err)
sps = DecodeSPS(b)
require.Equal(t, uint16(2560), sps.Width())
require.Equal(t, uint16(1920), sps.Height())
s = "Z2QAKKwa0AoAt03AQEBQAAADABAAAAMB6PFCKg==" // TP-Link
b, err = base64.StdEncoding.DecodeString(s)
require.Nil(t, err)
sps = DecodeSPS(b)
require.Equal(t, uint16(1280), sps.Width())
require.Equal(t, uint16(720), sps.Height())
s = "Z2QAFqwa0BQF/yzcBAQFAAADAAEAAAMAHo8UIqA=" // TP-Link sub
b, err = base64.StdEncoding.DecodeString(s)
require.Nil(t, err)
sps = DecodeSPS(b)
require.Equal(t, uint16(640), sps.Width())
require.Equal(t, uint16(360), sps.Height())
}
func TestGetProfileLevelID(t *testing.T) {
// OpenIPC https://github.com/OpenIPC
s := "profile-level-id=0033e7; packetization-mode=1; "
profile := GetProfileLevelID(s)
require.Equal(t, "640029", profile)
// Eufy T8400 https://github.com/AlexxIT/go2rtc/issues/155
s = "packetization-mode=1;profile-level-id=276400"
profile = GetProfileLevelID(s)
require.Equal(t, "640029", profile)
}
func TestDecodeSPS2(t *testing.T) {
s := "6764001fad84010c20086100430802184010c200843b50740932"
b, err := hex.DecodeString(s)
require.Nil(t, err)
sps := DecodeSPS(b)
require.Equal(t, uint16(928), sps.Width())
require.Equal(t, uint16(576), sps.Height())
s = "Z2QAHq2EAQwgCGEAQwgCGEAQwgCEO1BQF/yzcBAQFAAAD6AAAXcCEA==" // unknown
b, err = base64.StdEncoding.DecodeString(s)
require.Nil(t, err)
sps = DecodeSPS(b)
require.Equal(t, uint16(640), sps.Width())
require.Equal(t, uint16(360), sps.Height())
}
func TestAVCCToCodec(t *testing.T) {
s := "000000196764001fac2484014016ec0440000003004000000c23c60c920000000568ee32c8b0000000d365"
b, _ := hex.DecodeString(s)
codec := AVCCToCodec(b)
require.Equal(t, "packetization-mode=1;profile-level-id=64001f;sprop-parameter-sets=Z2QAH6wkhAFAFuwEQAAAAwBAAAAMI8YMkg==,aO4yyLA=", codec.FmtpLine)
}
+101
View File
@@ -0,0 +1,101 @@
// Package h264 - MPEG4 format related functions
package h264
import (
"bytes"
"encoding/base64"
"encoding/binary"
"encoding/hex"
"github.com/AlexxIT/go2rtc/pkg/core"
)
// DecodeConfig - extract profile, SPS and PPS from MPEG4 config
func DecodeConfig(conf []byte) (profile []byte, sps []byte, pps []byte) {
if len(conf) < 6 || conf[0] != 1 {
return
}
profile = conf[1:4]
count := conf[5] & 0x1F
conf = conf[6:]
for i := byte(0); i < count; i++ {
if len(conf) < 2 {
return
}
size := 2 + int(binary.BigEndian.Uint16(conf))
if len(conf) < size {
return
}
if sps == nil {
sps = conf[2:size]
}
conf = conf[size:]
}
count = conf[0]
conf = conf[1:]
for i := byte(0); i < count; i++ {
if len(conf) < 2 {
return
}
size := 2 + int(binary.BigEndian.Uint16(conf))
if len(conf) < size {
return
}
if pps == nil {
pps = conf[2:size]
}
conf = conf[size:]
}
return
}
func EncodeConfig(sps, pps []byte) []byte {
spsSize := uint16(len(sps))
ppsSize := uint16(len(pps))
buf := make([]byte, 5+3+spsSize+3+ppsSize)
buf[0] = 1
copy(buf[1:], sps[1:4]) // profile
buf[4] = 3 | 0xFC // ? LengthSizeMinusOne
b := buf[5:]
_ = b[3]
b[0] = 1 | 0xE0 // ? sps count
binary.BigEndian.PutUint16(b[1:], spsSize)
copy(b[3:], sps)
b = buf[5+3+spsSize:]
_ = b[3]
b[0] = 1 // pps count
binary.BigEndian.PutUint16(b[1:], ppsSize)
copy(b[3:], pps)
return buf
}
func ConfigToCodec(conf []byte) *core.Codec {
buf := bytes.NewBufferString("packetization-mode=1")
profile, sps, pps := DecodeConfig(conf)
if profile != nil {
buf.WriteString(";profile-level-id=")
buf.WriteString(hex.EncodeToString(profile))
}
if sps != nil && pps != nil {
buf.WriteString(";sprop-parameter-sets=")
buf.WriteString(base64.StdEncoding.EncodeToString(sps))
buf.WriteString(",")
buf.WriteString(base64.StdEncoding.EncodeToString(pps))
}
return &core.Codec{
Name: core.CodecH264,
ClockRate: 90000,
FmtpLine: buf.String(),
PayloadType: core.PayloadTypeRAW,
}
}
@@ -0,0 +1,195 @@
package h264
import "encoding/binary"
// Payloader payloads H264 packets
type Payloader struct {
IsAVC bool
stapANalu []byte
}
const (
stapaNALUType = 24
fuaNALUType = 28
fubNALUType = 29
spsNALUType = 7
ppsNALUType = 8
audNALUType = 9
fillerNALUType = 12
fuaHeaderSize = 2
//stapaHeaderSize = 1
//stapaNALULengthSize = 2
naluTypeBitmask = 0x1F
naluRefIdcBitmask = 0x60
//fuStartBitmask = 0x80
//fuEndBitmask = 0x40
outputStapAHeader = 0x78
)
//func annexbNALUStartCode() []byte { return []byte{0x00, 0x00, 0x00, 0x01} }
func EmitNalus(nals []byte, isAVC bool, emit func([]byte)) {
if !isAVC {
nextInd := func(nalu []byte, start int) (indStart int, indLen int) {
zeroCount := 0
for i, b := range nalu[start:] {
if b == 0 {
zeroCount++
continue
} else if b == 1 {
if zeroCount >= 2 {
return start + i - zeroCount, zeroCount + 1
}
}
zeroCount = 0
}
return -1, -1
}
nextIndStart, nextIndLen := nextInd(nals, 0)
if nextIndStart == -1 {
emit(nals)
} else {
for nextIndStart != -1 {
prevStart := nextIndStart + nextIndLen
nextIndStart, nextIndLen = nextInd(nals, prevStart)
if nextIndStart != -1 {
emit(nals[prevStart:nextIndStart])
} else {
// Emit until end of stream, no end indicator found
emit(nals[prevStart:])
}
}
}
} else {
for {
n := uint32(len(nals))
if n < 4 {
break
}
end := 4 + binary.BigEndian.Uint32(nals)
if n < end {
break
}
emit(nals[4:end])
nals = nals[end:]
}
}
}
// Payload fragments a H264 packet across one or more byte arrays
func (p *Payloader) Payload(mtu uint16, payload []byte) [][]byte {
var payloads [][]byte
if len(payload) == 0 {
return payloads
}
EmitNalus(payload, p.IsAVC, func(nalu []byte) {
if len(nalu) == 0 {
return
}
naluType := nalu[0] & naluTypeBitmask
naluRefIdc := nalu[0] & naluRefIdcBitmask
switch naluType {
case audNALUType, fillerNALUType:
return
case spsNALUType, ppsNALUType:
if p.stapANalu == nil {
p.stapANalu = []byte{outputStapAHeader}
}
p.stapANalu = append(p.stapANalu, byte(len(nalu)>>8), byte(len(nalu)))
p.stapANalu = append(p.stapANalu, nalu...)
return
}
if p.stapANalu != nil {
// Pack current NALU with SPS and PPS as STAP-A
// Supports multiple PPS in a row
if len(p.stapANalu) <= int(mtu) {
payloads = append(payloads, p.stapANalu)
}
p.stapANalu = nil
}
// Single NALU
if len(nalu) <= int(mtu) {
out := make([]byte, len(nalu))
copy(out, nalu)
payloads = append(payloads, out)
return
}
// FU-A
maxFragmentSize := int(mtu) - fuaHeaderSize
// The FU payload consists of fragments of the payload of the fragmented
// NAL unit so that if the fragmentation unit payloads of consecutive
// FUs are sequentially concatenated, the payload of the fragmented NAL
// unit can be reconstructed. The NAL unit type octet of the fragmented
// NAL unit is not included as such in the fragmentation unit payload,
// but rather the information of the NAL unit type octet of the
// fragmented NAL unit is conveyed in the F and NRI fields of the FU
// indicator octet of the fragmentation unit and in the type field of
// the FU header. An FU payload MAY have any number of octets and MAY
// be empty.
naluData := nalu
// According to the RFC, the first octet is skipped due to redundant information
naluDataIndex := 1
naluDataLength := len(nalu) - naluDataIndex
naluDataRemaining := naluDataLength
if min(maxFragmentSize, naluDataRemaining) <= 0 {
return
}
for naluDataRemaining > 0 {
currentFragmentSize := min(maxFragmentSize, naluDataRemaining)
out := make([]byte, fuaHeaderSize+currentFragmentSize)
// +---------------+
// |0|1|2|3|4|5|6|7|
// +-+-+-+-+-+-+-+-+
// |F|NRI| Type |
// +---------------+
out[0] = fuaNALUType
out[0] |= naluRefIdc
// +---------------+
// |0|1|2|3|4|5|6|7|
// +-+-+-+-+-+-+-+-+
// |S|E|R| Type |
// +---------------+
out[1] = naluType
if naluDataRemaining == naluDataLength {
// Set start bit
out[1] |= 1 << 7
} else if naluDataRemaining-currentFragmentSize == 0 {
// Set end bit
out[1] |= 1 << 6
}
copy(out[fuaHeaderSize:], naluData[naluDataIndex:naluDataIndex+currentFragmentSize])
payloads = append(payloads, out)
naluDataRemaining -= currentFragmentSize
naluDataIndex += currentFragmentSize
}
})
return payloads
}
func min(a, b int) int {
if a < b {
return a
}
return b
}
+137
View File
@@ -0,0 +1,137 @@
package h264
import (
"encoding/binary"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/h264/annexb"
"github.com/pion/rtp"
"github.com/pion/rtp/codecs"
)
const RTPPacketVersionAVC = 0
const PSMaxSize = 128 // the biggest SPS I've seen is 48 (EZVIZ CS-CV210)
func RTPDepay(codec *core.Codec, handler core.HandlerFunc) core.HandlerFunc {
depack := &codecs.H264Packet{IsAVC: true}
sps, pps := GetParameterSet(codec.FmtpLine)
ps := JoinNALU(sps, pps)
buf := make([]byte, 0, 512*1024) // 512K
return func(packet *rtp.Packet) {
//log.Printf("[RTP] codec: %s, nalu: %2d, size: %6d, ts: %10d, pt: %2d, ssrc: %d, seq: %d, %v", codec.Name, packet.Payload[0]&0x1F, len(packet.Payload), packet.Timestamp, packet.PayloadType, packet.SSRC, packet.SequenceNumber, packet.Marker)
payload, err := depack.Unmarshal(packet.Payload)
if len(payload) == 0 || err != nil {
return
}
// Memory overflow protection. Can happen if we miss a lot of packets with the marker.
// https://github.com/AlexxIT/go2rtc/issues/675
if len(buf) > 5*1024*1024 {
buf = buf[: 0 : 512*1024]
}
// Fix TP-Link Tapo TC70: sends SPS and PPS with packet.Marker = true
// Reolink Duo 2: sends SPS with Marker and PPS without
if packet.Marker && len(payload) < PSMaxSize {
switch NALUType(payload) {
case NALUTypeSPS, NALUTypePPS:
buf = append(buf, payload...)
return
case NALUTypeSEI:
// RtspServer https://github.com/AlexxIT/go2rtc/issues/244
// sends, marked SPS, marked PPS, marked SEI, marked IFrame
return
}
}
if len(buf) == 0 {
for {
// Amcrest IP4M-1051: 9, 7, 8, 6, 28...
// Amcrest IP4M-1051: 9, 6, 1
switch NALUType(payload) {
case NALUTypeIFrame:
// fix IFrame without SPS,PPS
buf = append(buf, ps...)
case NALUTypeSEI, NALUTypeAUD:
// fix ffmpeg with transcoding first frame
i := int(4 + binary.BigEndian.Uint32(payload))
// check if only one NAL (fix ffmpeg transcoding for Reolink RLC-510A)
if i == len(payload) {
return
}
payload = payload[i:]
continue
case NALUTypePFrame, NALUTypeSPS, NALUTypePPS: // pass
default:
return // skip any unknown NAL unit type
}
break
}
}
// collect all NALs for Access Unit
if !packet.Marker {
buf = append(buf, payload...)
return
}
if len(buf) > 0 {
payload = append(buf, payload...)
buf = buf[:0]
}
// should not be that huge SPS
if NALUType(payload) == NALUTypeSPS && binary.BigEndian.Uint32(payload) >= PSMaxSize {
// some Chinese buggy cameras have a single packet with SPS+PPS+IFrame separated by 00 00 00 01
// https://github.com/AlexxIT/WebRTC/issues/391
// https://github.com/AlexxIT/WebRTC/issues/392
payload = annexb.FixAnnexBInAVCC(payload)
}
//log.Printf("[AVC] %v, len: %d, ts: %10d, seq: %d", NALUTypes(payload), len(payload), packet.Timestamp, packet.SequenceNumber)
clone := *packet
clone.Version = RTPPacketVersionAVC
clone.Payload = payload
handler(&clone)
}
}
func RTPPay(mtu uint16, handler core.HandlerFunc) core.HandlerFunc {
if mtu == 0 {
mtu = 1472
}
payloader := &Payloader{IsAVC: true}
sequencer := rtp.NewRandomSequencer()
mtu -= 12 // rtp.Header size
return func(packet *rtp.Packet) {
if packet.Version != RTPPacketVersionAVC {
handler(packet)
return
}
payloads := payloader.Payload(mtu, packet.Payload)
last := len(payloads) - 1
for i, payload := range payloads {
clone := rtp.Packet{
Header: rtp.Header{
Version: 2,
Marker: i == last,
SequenceNumber: sequencer.NextSequenceNumber(),
Timestamp: packet.Timestamp,
},
Payload: payload,
}
handler(&clone)
}
}
}
+366
View File
@@ -0,0 +1,366 @@
package h264
import (
"fmt"
"github.com/AlexxIT/go2rtc/pkg/bits"
)
// http://www.itu.int/rec/T-REC-H.264
// https://webrtc.googlesource.com/src/+/refs/heads/main/common_video/h264/sps_parser.cc
//goland:noinspection GoSnakeCaseUsage
type SPS struct {
profile_idc uint8
profile_iop uint8
level_idc uint8
seq_parameter_set_id uint32
chroma_format_idc uint32
separate_colour_plane_flag byte
bit_depth_luma_minus8 uint32
bit_depth_chroma_minus8 uint32
qpprime_y_zero_transform_bypass_flag byte
seq_scaling_matrix_present_flag byte
log2_max_frame_num_minus4 uint32
pic_order_cnt_type uint32
log2_max_pic_order_cnt_lsb_minus4 uint32
delta_pic_order_always_zero_flag byte
offset_for_non_ref_pic int32
offset_for_top_to_bottom_field int32
num_ref_frames_in_pic_order_cnt_cycle uint32
num_ref_frames uint32
gaps_in_frame_num_value_allowed_flag byte
pic_width_in_mbs_minus_1 uint32
pic_height_in_map_units_minus_1 uint32
frame_mbs_only_flag byte
mb_adaptive_frame_field_flag byte
direct_8x8_inference_flag byte
frame_cropping_flag byte
frame_crop_left_offset uint32
frame_crop_right_offset uint32
frame_crop_top_offset uint32
frame_crop_bottom_offset uint32
vui_parameters_present_flag byte
aspect_ratio_info_present_flag byte
aspect_ratio_idc byte
sar_width uint16
sar_height uint16
overscan_info_present_flag byte
overscan_appropriate_flag byte
video_signal_type_present_flag byte
video_format uint8
video_full_range_flag byte
colour_description_present_flag byte
colour_description uint32
chroma_loc_info_present_flag byte
chroma_sample_loc_type_top_field uint32
chroma_sample_loc_type_bottom_field uint32
timing_info_present_flag byte
num_units_in_tick uint32
time_scale uint32
fixed_frame_rate_flag byte
}
func (s *SPS) Width() uint16 {
width := 16 * (s.pic_width_in_mbs_minus_1 + 1)
crop := 2 * (s.frame_crop_left_offset + s.frame_crop_right_offset)
return uint16(width - crop)
}
func (s *SPS) Height() uint16 {
height := 16 * (s.pic_height_in_map_units_minus_1 + 1)
crop := 2 * (s.frame_crop_top_offset + s.frame_crop_bottom_offset)
if s.frame_mbs_only_flag == 0 {
height *= 2
}
return uint16(height - crop)
}
func DecodeSPS(sps []byte) *SPS {
// https://developer.ridgerun.com/wiki/index.php/H264_Analysis_Tools
// ffmpeg -i file.h264 -c copy -bsf:v trace_headers -f null -
r := bits.NewReader(sps)
hdr := r.ReadByte()
if hdr&0x1F != NALUTypeSPS {
return nil
}
s := &SPS{
profile_idc: r.ReadByte(),
profile_iop: r.ReadByte(),
level_idc: r.ReadByte(),
seq_parameter_set_id: r.ReadUEGolomb(),
}
switch s.profile_idc {
case 100, 110, 122, 244, 44, 83, 86, 118, 128, 138, 139, 134, 135:
n := byte(8)
s.chroma_format_idc = r.ReadUEGolomb()
if s.chroma_format_idc == 3 {
s.separate_colour_plane_flag = r.ReadBit()
n = 12
}
s.bit_depth_luma_minus8 = r.ReadUEGolomb()
s.bit_depth_chroma_minus8 = r.ReadUEGolomb()
s.qpprime_y_zero_transform_bypass_flag = r.ReadBit()
s.seq_scaling_matrix_present_flag = r.ReadBit()
if s.seq_scaling_matrix_present_flag != 0 {
for i := byte(0); i < n; i++ {
//goland:noinspection GoSnakeCaseUsage
seq_scaling_list_present_flag := r.ReadBit()
if seq_scaling_list_present_flag != 0 {
if i < 6 {
s.scaling_list(r, 16)
} else {
s.scaling_list(r, 64)
}
}
}
}
}
s.log2_max_frame_num_minus4 = r.ReadUEGolomb()
s.pic_order_cnt_type = r.ReadUEGolomb()
switch s.pic_order_cnt_type {
case 0:
s.log2_max_pic_order_cnt_lsb_minus4 = r.ReadUEGolomb()
case 1:
s.delta_pic_order_always_zero_flag = r.ReadBit()
s.offset_for_non_ref_pic = r.ReadSEGolomb()
s.offset_for_top_to_bottom_field = r.ReadSEGolomb()
s.num_ref_frames_in_pic_order_cnt_cycle = r.ReadUEGolomb()
for i := uint32(0); i < s.num_ref_frames_in_pic_order_cnt_cycle; i++ {
_ = r.ReadSEGolomb() // offset_for_ref_frame[i]
}
}
s.num_ref_frames = r.ReadUEGolomb()
s.gaps_in_frame_num_value_allowed_flag = r.ReadBit()
s.pic_width_in_mbs_minus_1 = r.ReadUEGolomb()
s.pic_height_in_map_units_minus_1 = r.ReadUEGolomb()
s.frame_mbs_only_flag = r.ReadBit()
if s.frame_mbs_only_flag == 0 {
s.mb_adaptive_frame_field_flag = r.ReadBit()
}
s.direct_8x8_inference_flag = r.ReadBit()
s.frame_cropping_flag = r.ReadBit()
if s.frame_cropping_flag != 0 {
s.frame_crop_left_offset = r.ReadUEGolomb()
s.frame_crop_right_offset = r.ReadUEGolomb()
s.frame_crop_top_offset = r.ReadUEGolomb()
s.frame_crop_bottom_offset = r.ReadUEGolomb()
}
s.vui_parameters_present_flag = r.ReadBit()
if s.vui_parameters_present_flag != 0 {
s.aspect_ratio_info_present_flag = r.ReadBit()
if s.aspect_ratio_info_present_flag != 0 {
s.aspect_ratio_idc = r.ReadByte()
if s.aspect_ratio_idc == 255 {
s.sar_width = r.ReadUint16()
s.sar_height = r.ReadUint16()
}
}
s.overscan_info_present_flag = r.ReadBit()
if s.overscan_info_present_flag != 0 {
s.overscan_appropriate_flag = r.ReadBit()
}
s.video_signal_type_present_flag = r.ReadBit()
if s.video_signal_type_present_flag != 0 {
s.video_format = r.ReadBits8(3)
s.video_full_range_flag = r.ReadBit()
s.colour_description_present_flag = r.ReadBit()
if s.colour_description_present_flag != 0 {
s.colour_description = r.ReadUint24()
}
}
s.chroma_loc_info_present_flag = r.ReadBit()
if s.chroma_loc_info_present_flag != 0 {
s.chroma_sample_loc_type_top_field = r.ReadUEGolomb()
s.chroma_sample_loc_type_bottom_field = r.ReadUEGolomb()
}
s.timing_info_present_flag = r.ReadBit()
if s.timing_info_present_flag != 0 {
s.num_units_in_tick = r.ReadUint32()
s.time_scale = r.ReadUint32()
s.fixed_frame_rate_flag = r.ReadBit()
}
//...
}
if r.EOF {
return nil
}
return s
}
//goland:noinspection GoSnakeCaseUsage
func (s *SPS) scaling_list(r *bits.Reader, sizeOfScalingList int) {
lastScale := int32(8)
nextScale := int32(8)
for j := 0; j < sizeOfScalingList; j++ {
if nextScale != 0 {
delta_scale := r.ReadSEGolomb()
nextScale = (lastScale + delta_scale + 256) % 256
}
if nextScale != 0 {
lastScale = nextScale
}
}
}
func (s *SPS) Profile() string {
switch s.profile_idc {
case 0x42:
return "Baseline"
case 0x4D:
return "Main"
case 0x58:
return "Extended"
case 0x64:
return "High"
}
return fmt.Sprintf("0x%02X", s.profile_idc)
}
func (s *SPS) PixFmt() string {
if s.bit_depth_luma_minus8 == 0 {
switch s.chroma_format_idc {
case 1:
if s.video_full_range_flag == 1 {
return "yuvj420p"
}
return "yuv420p"
case 2:
return "yuv422p"
case 3:
return "yuv444p"
}
}
return ""
}
func (s *SPS) String() string {
return fmt.Sprintf(
"%s %d.%d, %s, %dx%d",
s.Profile(), s.level_idc/10, s.level_idc%10, s.PixFmt(), s.Width(), s.Height(),
)
}
// FixPixFmt - change yuvj420p to yuv420p in SPS
// same as "-c:v copy -bsf:v h264_metadata=video_full_range_flag=0"
func FixPixFmt(sps []byte) {
r := bits.NewReader(sps)
_ = r.ReadByte()
profile := r.ReadByte()
_ = r.ReadByte()
_ = r.ReadByte()
_ = r.ReadUEGolomb()
switch profile {
case 100, 110, 122, 244, 44, 83, 86, 118, 128, 138, 139, 134, 135:
n := byte(8)
if r.ReadUEGolomb() == 3 {
_ = r.ReadBit()
n = 12
}
_ = r.ReadUEGolomb()
_ = r.ReadUEGolomb()
_ = r.ReadBit()
if r.ReadBit() != 0 {
for i := byte(0); i < n; i++ {
if r.ReadBit() != 0 {
return // skip
}
}
}
}
_ = r.ReadUEGolomb()
switch r.ReadUEGolomb() {
case 0:
_ = r.ReadUEGolomb()
case 1:
_ = r.ReadBit()
_ = r.ReadSEGolomb()
_ = r.ReadSEGolomb()
n := r.ReadUEGolomb()
for i := uint32(0); i < n; i++ {
_ = r.ReadSEGolomb()
}
}
_ = r.ReadUEGolomb()
_ = r.ReadBit()
_ = r.ReadUEGolomb()
_ = r.ReadUEGolomb()
if r.ReadBit() == 0 {
_ = r.ReadBit()
}
_ = r.ReadBit()
if r.ReadBit() != 0 {
_ = r.ReadUEGolomb()
_ = r.ReadUEGolomb()
_ = r.ReadUEGolomb()
_ = r.ReadUEGolomb()
}
if r.ReadBit() != 0 {
if r.ReadBit() != 0 {
if r.ReadByte() == 255 {
_ = r.ReadUint16()
_ = r.ReadUint16()
}
}
if r.ReadBit() != 0 {
_ = r.ReadBit()
}
if r.ReadBit() != 0 {
_ = r.ReadBits8(3)
if r.ReadBit() == 1 {
pos, bit := r.Pos()
sps[pos] &= ^byte(1 << bit)
}
}
}
}
@@ -0,0 +1,8 @@
# H265
Payloader code taken from [pion](https://github.com/pion/rtp) library branch [h265](https://github.com/pion/rtp/tree/h265), because it's still not in release. Thanks to [@kevmo314](https://github.com/kevmo314).
## Useful links
- https://datatracker.ietf.org/doc/html/rfc7798
- [Add initial support for WebRTC HEVC](https://trac.webkit.org/changeset/259452/webkit)
+54
View File
@@ -0,0 +1,54 @@
package h265
import "github.com/AlexxIT/go2rtc/pkg/h264"
const forbiddenZeroBit = 0x80
const nalUnitType = 0x3F
// Deprecated: DecodeStream - find and return first AU in AVC format
// useful for processing live streams with unknown separator size
func DecodeStream(annexb []byte) ([]byte, int) {
startPos := -1
i := 0
for {
// search next separator
if i = h264.IndexFrom(annexb, []byte{0, 0, 1}, i); i < 0 {
break
}
// move i to next AU
if i += 3; i >= len(annexb) {
break
}
// check if AU type valid
octet := annexb[i]
if octet&forbiddenZeroBit != 0 {
continue
}
nalType := (octet >> 1) & nalUnitType
if startPos >= 0 {
switch nalType {
case NALUTypeVPS, NALUTypePFrame:
if annexb[i-4] == 0 {
return h264.DecodeAnnexB(annexb[startPos : i-4]), i - 4
} else {
return h264.DecodeAnnexB(annexb[startPos : i-3]), i - 3
}
}
} else {
switch nalType {
case NALUTypeVPS, NALUTypePFrame:
if i >= 4 && annexb[i-4] == 0 {
startPos = i - 4
} else {
startPos = i - 3
}
}
}
}
return nil, 0
}
+61
View File
@@ -0,0 +1,61 @@
// Package h265 - AVCC format related functions
package h265
import (
"bytes"
"encoding/base64"
"encoding/binary"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/h264"
"github.com/pion/rtp"
)
func RepairAVCC(codec *core.Codec, handler core.HandlerFunc) core.HandlerFunc {
vds, sps, pps := GetParameterSet(codec.FmtpLine)
ps := h264.JoinNALU(vds, sps, pps)
return func(packet *rtp.Packet) {
switch NALUType(packet.Payload) {
case NALUTypeIFrame, NALUTypeIFrame2, NALUTypeIFrame3:
clone := *packet
clone.Payload = h264.Join(ps, packet.Payload)
handler(&clone)
default:
handler(packet)
}
}
}
func AVCCToCodec(avcc []byte) *core.Codec {
buf := bytes.NewBufferString("profile-id=1")
for {
size := 4 + int(binary.BigEndian.Uint32(avcc))
switch NALUType(avcc) {
case NALUTypeVPS:
buf.WriteString(";sprop-vps=")
buf.WriteString(base64.StdEncoding.EncodeToString(avcc[4:size]))
case NALUTypeSPS:
buf.WriteString(";sprop-sps=")
buf.WriteString(base64.StdEncoding.EncodeToString(avcc[4:size]))
case NALUTypePPS:
buf.WriteString(";sprop-pps=")
buf.WriteString(base64.StdEncoding.EncodeToString(avcc[4:size]))
}
if size < len(avcc) {
avcc = avcc[size:]
} else {
break
}
}
return &core.Codec{
Name: core.CodecH265,
ClockRate: 90000,
FmtpLine: buf.String(),
PayloadType: core.PayloadTypeRAW,
}
}
@@ -0,0 +1,30 @@
package h265
import (
"encoding/base64"
"testing"
"github.com/stretchr/testify/require"
)
func TestDecodeSPS(t *testing.T) {
s := "QgEBAWAAAAMAAAMAAAMAAAMAmaAAoAgBaH+KrTuiS7/8AAQABbAgApMuADN/mAE="
b, err := base64.StdEncoding.DecodeString(s)
require.Nil(t, err)
sps := DecodeSPS(b)
require.NotNil(t, sps)
require.Equal(t, uint16(5120), sps.Width())
require.Equal(t, uint16(1440), sps.Height())
}
func TestDecodeSPS2(t *testing.T) {
s := "QgEBIUAAAAMAkAAAAwAAAwCWoAUCAWlnpbkShc1AQIC4QAAAAwBAAAAFFEn/eEAOpgAV+V8IBBA="
b, err := base64.StdEncoding.DecodeString(s)
require.Nil(t, err)
sps := DecodeSPS(b)
require.NotNil(t, sps)
require.Equal(t, uint16(640), sps.Width())
require.Equal(t, uint16(360), sps.Height())
}
@@ -0,0 +1,76 @@
package h265
import (
"encoding/base64"
"encoding/binary"
"github.com/AlexxIT/go2rtc/pkg/core"
)
const (
NALUTypePFrame = 1
NALUTypeIFrame = 19
NALUTypeIFrame2 = 20
NALUTypeIFrame3 = 21
NALUTypeVPS = 32
NALUTypeSPS = 33
NALUTypePPS = 34
NALUTypePrefixSEI = 39
NALUTypeSuffixSEI = 40
NALUTypeFU = 49
)
func NALUType(b []byte) byte {
return (b[4] >> 1) & 0x3F
}
func IsKeyframe(b []byte) bool {
for {
switch NALUType(b) {
case NALUTypePFrame:
return false
case NALUTypeIFrame, NALUTypeIFrame2, NALUTypeIFrame3:
return true
}
size := int(binary.BigEndian.Uint32(b)) + 4
if size < len(b) {
b = b[size:]
continue
} else {
return false
}
}
}
func Types(data []byte) []byte {
var types []byte
for {
types = append(types, NALUType(data))
size := 4 + int(binary.BigEndian.Uint32(data))
if size < len(data) {
data = data[size:]
} else {
break
}
}
return types
}
func GetParameterSet(fmtp string) (vps, sps, pps []byte) {
if fmtp == "" {
return
}
s := core.Between(fmtp, "sprop-vps=", ";")
vps, _ = base64.StdEncoding.DecodeString(s)
s = core.Between(fmtp, "sprop-sps=", ";")
sps, _ = base64.StdEncoding.DecodeString(s)
s = core.Between(fmtp, "sprop-pps=", ";")
pps, _ = base64.StdEncoding.DecodeString(s)
return
}
+98
View File
@@ -0,0 +1,98 @@
// Package h265 - MPEG4 format related functions
package h265
import (
"bytes"
"encoding/base64"
"encoding/binary"
"github.com/AlexxIT/go2rtc/pkg/core"
)
func DecodeConfig(conf []byte) (profile, vps, sps, pps []byte) {
profile = conf[1:4]
b := conf[23:]
if binary.BigEndian.Uint16(b[1:]) != 1 {
return
}
vpsSize := binary.BigEndian.Uint16(b[3:])
vps = b[5 : 5+vpsSize]
b = conf[23+5+vpsSize:]
if binary.BigEndian.Uint16(b[1:]) != 1 {
return
}
spsSize := binary.BigEndian.Uint16(b[3:])
sps = b[5 : 5+spsSize]
b = conf[23+5+vpsSize+5+spsSize:]
if binary.BigEndian.Uint16(b[1:]) != 1 {
return
}
ppsSize := binary.BigEndian.Uint16(b[3:])
pps = b[5 : 5+ppsSize]
return
}
func EncodeConfig(vps, sps, pps []byte) []byte {
vpsSize := uint16(len(vps))
spsSize := uint16(len(sps))
ppsSize := uint16(len(pps))
buf := make([]byte, 23+5+vpsSize+5+spsSize+5+ppsSize)
buf[0] = 1
copy(buf[1:], sps[3:6]) // profile
buf[21] = 3 // ?
buf[22] = 3 // ?
b := buf[23:]
_ = b[5]
b[0] = (vps[0] >> 1) & 0x3F
binary.BigEndian.PutUint16(b[1:], 1) // VPS count
binary.BigEndian.PutUint16(b[3:], vpsSize)
copy(b[5:], vps)
b = buf[23+5+vpsSize:]
_ = b[5]
b[0] = (sps[0] >> 1) & 0x3F
binary.BigEndian.PutUint16(b[1:], 1) // SPS count
binary.BigEndian.PutUint16(b[3:], spsSize)
copy(b[5:], sps)
b = buf[23+5+vpsSize+5+spsSize:]
_ = b[5]
b[0] = (pps[0] >> 1) & 0x3F
binary.BigEndian.PutUint16(b[1:], 1) // PPS count
binary.BigEndian.PutUint16(b[3:], ppsSize)
copy(b[5:], pps)
return buf
}
func ConfigToCodec(conf []byte) *core.Codec {
buf := bytes.NewBufferString("profile-id=1")
_, vps, sps, pps := DecodeConfig(conf)
if vps != nil {
buf.WriteString(";sprop-vps=")
buf.WriteString(base64.StdEncoding.EncodeToString(vps))
}
if sps != nil {
buf.WriteString(";sprop-sps=")
buf.WriteString(base64.StdEncoding.EncodeToString(sps))
}
if pps != nil {
buf.WriteString(";sprop-pps=")
buf.WriteString(base64.StdEncoding.EncodeToString(pps))
}
return &core.Codec{
Name: core.CodecH265,
ClockRate: 90000,
FmtpLine: buf.String(),
PayloadType: core.PayloadTypeRAW,
}
}
@@ -0,0 +1,301 @@
package h265
import (
"encoding/binary"
"math"
"github.com/AlexxIT/go2rtc/pkg/h264"
)
//
// Network Abstraction Unit Header implementation
//
const (
// sizeof(uint16)
h265NaluHeaderSize = 2
// https://datatracker.ietf.org/doc/html/rfc7798#section-4.4.2
h265NaluAggregationPacketType = 48
// https://datatracker.ietf.org/doc/html/rfc7798#section-4.4.3
h265NaluFragmentationUnitType = 49
// https://datatracker.ietf.org/doc/html/rfc7798#section-4.4.4
h265NaluPACIPacketType = 50
)
// H265NALUHeader is a H265 NAL Unit Header
// https://datatracker.ietf.org/doc/html/rfc7798#section-1.1.4
// +---------------+---------------+
//
// |0|1|2|3|4|5|6|7|0|1|2|3|4|5|6|7|
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// |F| Type | LayerID | TID |
// +-------------+-----------------+
type H265NALUHeader uint16
func newH265NALUHeader(highByte, lowByte uint8) H265NALUHeader {
return H265NALUHeader((uint16(highByte) << 8) | uint16(lowByte))
}
// F is the forbidden bit, should always be 0.
func (h H265NALUHeader) F() bool {
return (uint16(h) >> 15) != 0
}
// Type of NAL Unit.
func (h H265NALUHeader) Type() uint8 {
// 01111110 00000000
const mask = 0b01111110 << 8
return uint8((uint16(h) & mask) >> (8 + 1))
}
// IsTypeVCLUnit returns whether or not the NAL Unit type is a VCL NAL unit.
func (h H265NALUHeader) IsTypeVCLUnit() bool {
// Type is coded on 6 bits
const msbMask = 0b00100000
return (h.Type() & msbMask) == 0
}
// LayerID should always be 0 in non-3D HEVC context.
func (h H265NALUHeader) LayerID() uint8 {
// 00000001 11111000
const mask = (0b00000001 << 8) | 0b11111000
return uint8((uint16(h) & mask) >> 3)
}
// TID is the temporal identifier of the NAL unit +1.
func (h H265NALUHeader) TID() uint8 {
const mask = 0b00000111
return uint8(uint16(h) & mask)
}
// IsAggregationPacket returns whether or not the packet is an Aggregation packet.
func (h H265NALUHeader) IsAggregationPacket() bool {
return h.Type() == h265NaluAggregationPacketType
}
// IsFragmentationUnit returns whether or not the packet is a Fragmentation Unit packet.
func (h H265NALUHeader) IsFragmentationUnit() bool {
return h.Type() == h265NaluFragmentationUnitType
}
// IsPACIPacket returns whether or not the packet is a PACI packet.
func (h H265NALUHeader) IsPACIPacket() bool {
return h.Type() == h265NaluPACIPacketType
}
//
// Fragmentation Unit implementation
//
const (
// sizeof(uint8)
h265FragmentationUnitHeaderSize = 1
)
// H265FragmentationUnitHeader is a H265 FU Header
// +---------------+
// |0|1|2|3|4|5|6|7|
// +-+-+-+-+-+-+-+-+
// |S|E| FuType |
// +---------------+
type H265FragmentationUnitHeader uint8
// S represents the start of a fragmented NAL unit.
func (h H265FragmentationUnitHeader) S() bool {
const mask = 0b10000000
return ((h & mask) >> 7) != 0
}
// E represents the end of a fragmented NAL unit.
func (h H265FragmentationUnitHeader) E() bool {
const mask = 0b01000000
return ((h & mask) >> 6) != 0
}
// FuType MUST be equal to the field Type of the fragmented NAL unit.
func (h H265FragmentationUnitHeader) FuType() uint8 {
const mask = 0b00111111
return uint8(h) & mask
}
// Payloader payloads H265 packets
type Payloader struct {
AddDONL bool
SkipAggregation bool
donl uint16
}
// Payload fragments a H265 packet across one or more byte arrays
func (p *Payloader) Payload(mtu uint16, payload []byte) [][]byte {
var payloads [][]byte
if len(payload) == 0 {
return payloads
}
bufferedNALUs := make([][]byte, 0)
aggregationBufferSize := 0
flushBufferedNals := func() {
if len(bufferedNALUs) == 0 {
return
}
if len(bufferedNALUs) == 1 {
// emit this as a single NALU packet
nalu := bufferedNALUs[0]
if p.AddDONL {
buf := make([]byte, len(nalu)+2)
// copy the NALU header to the payload header
copy(buf[0:h265NaluHeaderSize], nalu[0:h265NaluHeaderSize])
// copy the DONL into the header
binary.BigEndian.PutUint16(buf[h265NaluHeaderSize:h265NaluHeaderSize+2], p.donl)
// write the payload
copy(buf[h265NaluHeaderSize+2:], nalu[h265NaluHeaderSize:])
p.donl++
payloads = append(payloads, buf)
} else {
// write the nalu directly to the payload
payloads = append(payloads, nalu)
}
} else {
// construct an aggregation packet
aggregationPacketSize := aggregationBufferSize + 2
buf := make([]byte, aggregationPacketSize)
layerID := uint8(math.MaxUint8)
tid := uint8(math.MaxUint8)
for _, nalu := range bufferedNALUs {
header := newH265NALUHeader(nalu[0], nalu[1])
headerLayerID := header.LayerID()
headerTID := header.TID()
if headerLayerID < layerID {
layerID = headerLayerID
}
if headerTID < tid {
tid = headerTID
}
}
binary.BigEndian.PutUint16(buf[0:2], (uint16(h265NaluAggregationPacketType)<<9)|(uint16(layerID)<<3)|uint16(tid))
index := 2
for i, nalu := range bufferedNALUs {
if p.AddDONL {
if i == 0 {
binary.BigEndian.PutUint16(buf[index:index+2], p.donl)
index += 2
} else {
buf[index] = byte(i - 1)
index++
}
}
binary.BigEndian.PutUint16(buf[index:index+2], uint16(len(nalu)))
index += 2
index += copy(buf[index:], nalu)
}
payloads = append(payloads, buf)
}
// clear the buffered NALUs
bufferedNALUs = make([][]byte, 0)
aggregationBufferSize = 0
}
h264.EmitNalus(payload, true, func(nalu []byte) {
if len(nalu) == 0 {
return
}
if len(nalu) <= int(mtu) {
// this nalu fits into a single packet, either it can be emitted as
// a single nalu or appended to the previous aggregation packet
marginalAggregationSize := len(nalu) + 2
if p.AddDONL {
marginalAggregationSize += 1
}
if aggregationBufferSize+marginalAggregationSize > int(mtu) {
flushBufferedNals()
}
bufferedNALUs = append(bufferedNALUs, nalu)
aggregationBufferSize += marginalAggregationSize
if p.SkipAggregation {
// emit this immediately.
flushBufferedNals()
}
} else {
// if this nalu doesn't fit in the current mtu, it needs to be fragmented
fuPacketHeaderSize := h265FragmentationUnitHeaderSize + 2 /* payload header size */
if p.AddDONL {
fuPacketHeaderSize += 2
}
// then, fragment the nalu
maxFUPayloadSize := int(mtu) - fuPacketHeaderSize
naluHeader := newH265NALUHeader(nalu[0], nalu[1])
// the nalu header is omitted from the fragmentation packet payload
nalu = nalu[h265NaluHeaderSize:]
if maxFUPayloadSize == 0 || len(nalu) == 0 {
return
}
// flush any buffered aggregation packets.
flushBufferedNals()
fullNALUSize := len(nalu)
for len(nalu) > 0 {
curentFUPayloadSize := len(nalu)
if curentFUPayloadSize > maxFUPayloadSize {
curentFUPayloadSize = maxFUPayloadSize
}
out := make([]byte, fuPacketHeaderSize+curentFUPayloadSize)
// write the payload header
binary.BigEndian.PutUint16(out[0:2], uint16(naluHeader))
out[0] = (out[0] & 0b10000001) | h265NaluFragmentationUnitType<<1
// write the fragment header
out[2] = byte(H265FragmentationUnitHeader(naluHeader.Type()))
if len(nalu) == fullNALUSize {
// Set start bit
out[2] |= 1 << 7
} else if len(nalu)-curentFUPayloadSize == 0 {
// Set end bit
out[2] |= 1 << 6
}
if p.AddDONL {
// write the DONL header
binary.BigEndian.PutUint16(out[3:5], p.donl)
p.donl++
// copy the fragment payload
copy(out[5:], nalu[0:curentFUPayloadSize])
} else {
// copy the fragment payload
copy(out[3:], nalu[0:curentFUPayloadSize])
}
// append the fragment to the payload
payloads = append(payloads, out)
// advance the nalu data pointer
nalu = nalu[curentFUPayloadSize:]
}
}
})
flushBufferedNals()
return payloads
}
+221
View File
@@ -0,0 +1,221 @@
package h265
import (
"encoding/binary"
"github.com/AlexxIT/go2rtc/pkg/core"
"github.com/AlexxIT/go2rtc/pkg/h264"
"github.com/pion/rtp"
)
func RTPDepay(codec *core.Codec, handler core.HandlerFunc) core.HandlerFunc {
vps, sps, pps := GetParameterSet(codec.FmtpLine)
ps := h264.JoinNALU(vps, sps, pps)
buf := make([]byte, 0, 512*1024) // 512K
var nuStart int
var seqNum uint16
return func(packet *rtp.Packet) {
data := packet.Payload
if len(data) < 3 {
return
}
nuType := (data[0] >> 1) & 0x3F
//log.Printf("[RTP] codec: %s, nalu: %2d, size: %6d, ts: %10d, pt: %2d, ssrc: %d, seq: %d, %v", track.Codec.Name, nuType, len(packet.Payload), packet.Timestamp, packet.PayloadType, packet.SSRC, packet.SequenceNumber, packet.Marker)
// Fix for RtspServer https://github.com/AlexxIT/go2rtc/issues/244
if packet.Marker && len(data) < h264.PSMaxSize {
switch nuType {
case NALUTypeVPS, NALUTypeSPS, NALUTypePPS:
packet.Marker = false
case NALUTypePrefixSEI, NALUTypeSuffixSEI:
return
}
}
// when we collect data into one buffer, we need to make sure
// that all of it falls into the same sequence
if len(buf) > 0 && packet.SequenceNumber-seqNum != 1 {
//log.Printf("broken H265 sequence")
buf = buf[:0] // drop data
return
}
seqNum = packet.SequenceNumber
if nuType == NALUTypeFU {
switch data[2] >> 6 {
case 0b10: // begin
nuType = data[2] & 0x3F
// push PS data before keyframe
if len(buf) == 0 && nuType >= 19 && nuType <= 21 {
buf = append(buf, ps...)
}
nuStart = len(buf)
buf = append(buf, 0, 0, 0, 0) // NAL unit size
buf = append(buf, (data[0]&0x81)|(nuType<<1), data[1])
buf = append(buf, data[3:]...)
return
case 0b00: // continue
if len(buf) == 0 {
//log.Printf("broken H265 fragment")
return
}
buf = append(buf, data[3:]...)
return
case 0b01: // end
if len(buf) == 0 {
//log.Printf("broken H265 fragment")
return
}
buf = append(buf, data[3:]...)
if nuStart > len(buf)+4 {
//log.Printf("broken H265 fragment")
buf = buf[:0] // drop data
return
}
binary.BigEndian.PutUint32(buf[nuStart:], uint32(len(buf)-nuStart-4))
case 0b11: // wrong RFC 7798 realisation from OpenIPC project
// A non-fragmented NAL unit MUST NOT be transmitted in one FU; i.e.,
// the Start bit and End bit must not both be set to 1 in the same FU
// header.
nuType = data[2] & 0x3F
buf = binary.BigEndian.AppendUint32(buf, uint32(len(data))-1) // NAL unit size
buf = append(buf, (data[0]&0x81)|(nuType<<1), data[1])
buf = append(buf, data[3:]...)
}
} else {
buf = binary.BigEndian.AppendUint32(buf, uint32(len(data))) // NAL unit size
buf = append(buf, data...)
}
// collect all NAL Units for Access Unit
if !packet.Marker {
return
}
//log.Printf("[HEVC] %v, len: %d", Types(buf), len(buf))
clone := *packet
clone.Version = h264.RTPPacketVersionAVC
clone.Payload = buf
buf = buf[:0]
handler(&clone)
}
}
func RTPPay(mtu uint16, handler core.HandlerFunc) core.HandlerFunc {
if mtu == 0 {
mtu = 1472
}
payloader := &Payloader{}
sequencer := rtp.NewRandomSequencer()
mtu -= 12 // rtp.Header size
return func(packet *rtp.Packet) {
if packet.Version != h264.RTPPacketVersionAVC {
handler(packet)
return
}
payloads := payloader.Payload(mtu, packet.Payload)
last := len(payloads) - 1
for i, payload := range payloads {
clone := rtp.Packet{
Header: rtp.Header{
Version: 2,
Marker: i == last,
SequenceNumber: sequencer.NextSequenceNumber(),
Timestamp: packet.Timestamp,
},
Payload: payload,
}
handler(&clone)
}
}
}
// SafariPay - generate Safari friendly payload for H265
// https://github.com/AlexxIT/Blog/issues/5
func SafariPay(mtu uint16, handler core.HandlerFunc) core.HandlerFunc {
sequencer := rtp.NewRandomSequencer()
size := int(mtu - 12) // rtp.Header size
return func(packet *rtp.Packet) {
if packet.Version != h264.RTPPacketVersionAVC {
handler(packet)
return
}
// protect original packets from modification
au := make([]byte, len(packet.Payload))
copy(au, packet.Payload)
var start byte
for i := 0; i < len(au); {
size := int(binary.BigEndian.Uint32(au[i:])) + 4
// convert AVC to Annex-B
au[i] = 0
au[i+1] = 0
au[i+2] = 0
au[i+3] = 1
switch NALUType(au[i:]) {
case NALUTypeIFrame, NALUTypeIFrame2, NALUTypeIFrame3:
start = 3
default:
if start == 0 {
start = 2
}
}
i += size
}
// rtp.Packet payload
b := make([]byte, 1, size)
size-- // minus header byte
for au != nil {
b[0] = start
if start > 1 {
start -= 2
}
if len(au) > size {
b = append(b, au[:size]...)
au = au[size:]
} else {
b = append(b, au...)
au = nil
}
clone := rtp.Packet{
Header: rtp.Header{
Version: 2,
Marker: au == nil,
SequenceNumber: sequencer.NextSequenceNumber(),
Timestamp: packet.Timestamp,
},
Payload: b,
}
handler(&clone)
b = b[:1] // clear buffer
}
}
}
+126
View File
@@ -0,0 +1,126 @@
package h265
import (
"bytes"
"github.com/AlexxIT/go2rtc/pkg/bits"
)
// http://www.itu.int/rec/T-REC-H.265
//goland:noinspection GoSnakeCaseUsage
type SPS struct {
sps_video_parameter_set_id uint8
sps_max_sub_layers_minus1 uint8
sps_temporal_id_nesting_flag byte
general_profile_space uint8
general_tier_flag byte
general_profile_idc uint8
general_profile_compatibility_flags uint32
general_level_idc uint8
sub_layer_profile_present_flag []byte
sub_layer_level_present_flag []byte
sps_seq_parameter_set_id uint32
chroma_format_idc uint32
separate_colour_plane_flag byte
pic_width_in_luma_samples uint32
pic_height_in_luma_samples uint32
}
func (s *SPS) Width() uint16 {
return uint16(s.pic_width_in_luma_samples)
}
func (s *SPS) Height() uint16 {
return uint16(s.pic_height_in_luma_samples)
}
func DecodeSPS(nalu []byte) *SPS {
rbsp := bytes.ReplaceAll(nalu[2:], []byte{0, 0, 3}, []byte{0, 0})
r := bits.NewReader(rbsp)
s := &SPS{}
s.sps_video_parameter_set_id = r.ReadBits8(4)
s.sps_max_sub_layers_minus1 = r.ReadBits8(3)
s.sps_temporal_id_nesting_flag = r.ReadBit()
if !s.profile_tier_level(r) {
return nil
}
s.sps_seq_parameter_set_id = r.ReadUEGolomb()
s.chroma_format_idc = r.ReadUEGolomb()
if s.chroma_format_idc == 3 {
s.separate_colour_plane_flag = r.ReadBit()
}
s.pic_width_in_luma_samples = r.ReadUEGolomb()
s.pic_height_in_luma_samples = r.ReadUEGolomb()
//...
if r.EOF {
return nil
}
return s
}
// profile_tier_level supports ONLY general_profile_idc == 1
// over variants very complicated...
//
//goland:noinspection GoSnakeCaseUsage
func (s *SPS) profile_tier_level(r *bits.Reader) bool {
s.general_profile_space = r.ReadBits8(2)
s.general_tier_flag = r.ReadBit()
s.general_profile_idc = r.ReadBits8(5)
s.general_profile_compatibility_flags = r.ReadBits(32)
_ = r.ReadBits64(48) // other flags
if s.general_profile_idc != 1 {
return false
}
s.general_level_idc = r.ReadBits8(8)
s.sub_layer_profile_present_flag = make([]byte, s.sps_max_sub_layers_minus1)
s.sub_layer_level_present_flag = make([]byte, s.sps_max_sub_layers_minus1)
for i := byte(0); i < s.sps_max_sub_layers_minus1; i++ {
s.sub_layer_profile_present_flag[i] = r.ReadBit()
s.sub_layer_level_present_flag[i] = r.ReadBit()
}
if s.sps_max_sub_layers_minus1 > 0 {
for i := s.sps_max_sub_layers_minus1; i < 8; i++ {
_ = r.ReadBits8(2) // reserved_zero_2bits
}
}
for i := byte(0); i < s.sps_max_sub_layers_minus1; i++ {
if s.sub_layer_profile_present_flag[i] != 0 {
_ = r.ReadBits8(2) // sub_layer_profile_space
_ = r.ReadBit() // sub_layer_tier_flag
sub_layer_profile_idc := r.ReadBits8(5) // sub_layer_profile_idc
_ = r.ReadBits(32) // sub_layer_profile_compatibility_flag
_ = r.ReadBits64(48) // other flags
if sub_layer_profile_idc != 1 {
return false
}
}
if s.sub_layer_level_present_flag[i] != 0 {
_ = r.ReadBits8(8)
}
}
return true
}
+54
View File
@@ -0,0 +1,54 @@
# Home Accessory Protocol
> PS. Character = Characteristic
**Device** - HomeKit end device (swith, camera, etc)
- mDNS name: `MyCamera._hap._tcp.local.`
- DeviceID - mac-like: `0E:AA:CE:2B:35:71`
- HomeKit device is described by:
- one or more `Accessories` - has `AID` and `Services`
- `Services` - has `IID`, `Type` and `Characters`
- `Characters` - has `IID`, `Type`, `Format` and `Value`
**Client** - HomeKit client (iPhone, iPad, MacBook or opensource library)
- ClientID - static random UUID
- ClientPublic/ClientPrivate - static random 32 byte keypair
- can pair with Device (exchange ClientID/ClientPublic, ServerID/ServerPublic using Pin)
- can auth to Device using ClientPrivate
- holding persistant Secure connection to device
- can read device Accessories
- can read and write device Characters
- can subscribe on device Characters change (Event)
**Server** - HomeKit server (soft on end device or opensource library)
- ServerID - same as DeviceID (using for Client auth)
- ServerPublic/ServerPrivate - static random 32 byte keypair
## AAC ELD
Requires ffmpeg built with `--enable-libfdk-aac`
```
-acodec libfdk_aac -aprofile aac_eld
```
| SampleRate | RTPTime | constantDuration | objectType |
|------------|---------|--------------------|--------------|
| 8000 | 60 | =8000/1000*60=480 | 39 (AAC ELD) |
| 16000 | 30 | =16000/1000*30=480 | 39 (AAC ELD) |
| 24000 | 20 | =24000/1000*20=480 | 39 (AAC ELD) |
| 16000 | 60 | =16000/1000*60=960 | 23 (AAC LD) |
| 24000 | 40 | =24000/1000*40=960 | 23 (AAC LD) |
## Useful links
- https://github.com/apple/HomeKitADK/blob/master/Documentation/crypto.md
- https://github.com/apple/HomeKitADK/blob/master/HAP/HAPPairingPairSetup.c
- [Extracting HomeKit Pairing Keys](https://pvieito.com/2019/12/extract-homekit-pairing-keys)
- [HAP in AirPlay2 receiver](https://github.com/openairplay/airplay2-receiver/blob/master/ap2/pairing/hap.py)
- [HomeKit Secure Video Unofficial Specification](https://github.com/Supereg/secure-video-specification)
- [Homebridge Camera FFmpeg](https://sunoo.github.io/homebridge-camera-ffmpeg/configs/)
- https://github.com/ljezny/Particle-HAP/blob/master/HAP-Specification-Non-Commercial-Version.pdf
@@ -0,0 +1,176 @@
package hap
import (
"fmt"
"strconv"
)
const (
FormatString = "string"
FormatBool = "bool"
FormatFloat = "float"
FormatUInt8 = "uint8"
FormatUInt16 = "uint16"
FormatUInt32 = "uint32"
FormatInt32 = "int32"
FormatUInt64 = "uint64"
FormatData = "data"
FormatTLV8 = "tlv8"
UnitPercentage = "percentage"
)
var PR = []string{"pr"}
var PW = []string{"pw"}
var PRPW = []string{"pr", "pw"}
var EVPRPW = []string{"ev", "pr", "pw"}
var EVPR = []string{"ev", "pr"}
type Accessory struct {
AID uint8 `json:"aid"` // 150 unique accessories per bridge
Services []*Service `json:"services"`
}
func (a *Accessory) InitIID() {
serviceN := map[string]byte{}
for _, service := range a.Services {
if len(service.Type) > 3 {
panic(service.Type)
}
n := serviceN[service.Type] + 1
serviceN[service.Type] = n
if n > 15 {
panic(n)
}
// ServiceID = ANSSS000
s := fmt.Sprintf("%x%x%03s000", a.AID, n, service.Type)
service.IID, _ = strconv.ParseUint(s, 16, 64)
for _, character := range service.Characters {
if len(character.Type) > 3 {
panic(character.Type)
}
// CharacterID = ANSSSCCC
character.IID, _ = strconv.ParseUint(character.Type, 16, 64)
character.IID += service.IID
}
}
}
func (a *Accessory) GetService(servType string) *Service {
for _, serv := range a.Services {
if serv.Type == servType {
return serv
}
}
return nil
}
func (a *Accessory) GetCharacter(charType string) *Character {
for _, serv := range a.Services {
for _, char := range serv.Characters {
if char.Type == charType {
return char
}
}
}
return nil
}
func (a *Accessory) GetCharacterByID(iid uint64) *Character {
for _, serv := range a.Services {
for _, char := range serv.Characters {
if char.IID == iid {
return char
}
}
}
return nil
}
type Service struct {
Desc string `json:"description,omitempty"`
Type string `json:"type"`
IID uint64 `json:"iid"`
Primary bool `json:"primary,omitempty"`
Characters []*Character `json:"characteristics"`
Linked []int `json:"linked,omitempty"`
}
func (s *Service) GetCharacter(charType string) *Character {
for _, char := range s.Characters {
if char.Type == charType {
return char
}
}
return nil
}
func ServiceAccessoryInformation(manuf, model, name, serial, firmware string) *Service {
return &Service{
Type: "3E", // AccessoryInformation
Characters: []*Character{
{
Type: "14",
Format: FormatBool,
Perms: PW,
//Descr: "Identify",
}, {
Type: "20",
Format: FormatString,
Value: manuf,
Perms: PR,
//Descr: "Manufacturer",
//MaxLen: 64,
}, {
Type: "21",
Format: FormatString,
Value: model,
Perms: PR,
//Descr: "Model",
//MaxLen: 64,
}, {
Type: "23",
Format: FormatString,
Value: name,
Perms: PR,
//Descr: "Name",
//MaxLen: 64,
}, {
Type: "30",
Format: FormatString,
Value: serial,
Perms: PR,
//Descr: "Serial Number",
//MaxLen: 64,
}, {
Type: "52",
Format: FormatString,
Value: firmware,
Perms: PR,
//Descr: "Firmware Revision",
},
},
}
}
func ServiceHAPProtocolInformation() *Service {
return &Service{
Type: "A2", // 'HAPProtocolInformation'
Characters: []*Character{
{
Type: "37",
Format: FormatString,
Value: "1.1.0",
Perms: PR,
//Descr: "Version",
//MaxLen: 64,
},
},
}
}
@@ -0,0 +1,3 @@
## Useful links
- https://github.com/bauer-andreas/secure-video-specification
@@ -0,0 +1,149 @@
package camera
import (
"github.com/AlexxIT/go2rtc/pkg/hap"
"github.com/AlexxIT/go2rtc/pkg/hap/tlv8"
)
func NewAccessory(manuf, model, name, serial, firmware string) *hap.Accessory {
acc := &hap.Accessory{
AID: hap.DeviceAID,
Services: []*hap.Service{
hap.ServiceAccessoryInformation(manuf, model, name, serial, firmware),
ServiceCameraRTPStreamManagement(),
//hap.ServiceHAPProtocolInformation(),
ServiceMicrophone(),
},
}
acc.InitIID()
return acc
}
func ServiceMicrophone() *hap.Service {
return &hap.Service{
Type: "112", // 'Microphone'
Characters: []*hap.Character{
{
Type: "11A",
Format: hap.FormatBool,
Value: 0,
Perms: hap.EVPRPW,
//Descr: "Mute",
},
//{
// Type: "119",
// Format: hap.FormatUInt8,
// Value: 100,
// Perms: hap.EVPRPW,
// //Descr: "Volume",
// //Unit: hap.UnitPercentage,
// //MinValue: 0,
// //MaxValue: 100,
// //MinStep: 1,
//},
},
}
}
func ServiceCameraRTPStreamManagement() *hap.Service {
val120, _ := tlv8.MarshalBase64(StreamingStatus{
Status: StreamingStatusAvailable,
})
val114, _ := tlv8.MarshalBase64(SupportedVideoStreamConfiguration{
Codecs: []VideoCodecConfiguration{
{
CodecType: VideoCodecTypeH264,
CodecParams: []VideoCodecParameters{
{
ProfileID: []byte{VideoCodecProfileMain},
Level: []byte{VideoCodecLevel31, VideoCodecLevel40},
},
},
VideoAttrs: []VideoCodecAttributes{
{Width: 1920, Height: 1080, Framerate: 30},
{Width: 1280, Height: 720, Framerate: 30}, // important for iPhones
{Width: 320, Height: 240, Framerate: 15}, // apple watch
},
},
},
})
val115, _ := tlv8.MarshalBase64(SupportedAudioStreamConfiguration{
Codecs: []AudioCodecConfiguration{
{
CodecType: AudioCodecTypeOpus,
CodecParams: []AudioCodecParameters{
{
Channels: 1,
BitrateMode: AudioCodecBitrateVariable,
SampleRate: []byte{AudioCodecSampleRate16Khz},
},
},
},
},
ComfortNoiseSupport: 0,
})
val116, _ := tlv8.MarshalBase64(SupportedRTPConfiguration{
SRTPCryptoType: []byte{CryptoAES_CM_128_HMAC_SHA1_80},
})
service := &hap.Service{
Type: "110", // 'CameraRTPStreamManagement'
Characters: []*hap.Character{
{
Type: TypeStreamingStatus,
Format: hap.FormatTLV8,
Value: val120,
Perms: hap.EVPR,
//Descr: "Streaming Status",
},
{
Type: TypeSupportedVideoStreamConfiguration,
Format: hap.FormatTLV8,
Value: val114,
Perms: hap.PR,
//Descr: "Supported Video Stream Configuration",
},
{
Type: TypeSupportedAudioStreamConfiguration,
Format: hap.FormatTLV8,
Value: val115,
Perms: hap.PR,
//Descr: "Supported Audio Stream Configuration",
},
{
Type: TypeSupportedRTPConfiguration,
Format: hap.FormatTLV8,
Value: val116,
Perms: hap.PR,
//Descr: "Supported RTP Configuration",
},
{
Type: "B0",
Format: hap.FormatUInt8,
Value: 1,
Perms: hap.EVPRPW,
//Descr: "Active",
//MinValue: 0,
//MaxValue: 1,
//MinStep: 1,
//ValidVal: []any{0, 1},
},
{
Type: TypeSelectedStreamConfiguration,
Format: hap.FormatTLV8,
Value: "", // important empty
Perms: hap.PRPW,
//Descr: "Selected RTP Stream Configuration",
},
{
Type: TypeSetupEndpoints,
Format: hap.FormatTLV8,
Value: "", // important empty
Perms: hap.PRPW,
//Descr: "Setup Endpoints",
},
},
}
return service
}
@@ -0,0 +1,254 @@
package camera
import (
"encoding/base64"
"strings"
"testing"
"github.com/AlexxIT/go2rtc/pkg/hap"
"github.com/stretchr/testify/require"
)
func TestNilCharacter(t *testing.T) {
var res SetupEndpoints
char := &hap.Character{}
err := char.ReadTLV8(&res)
require.NotNil(t, err)
require.NotNil(t, strings.Contains(err.Error(), "can't read value"))
}
type testTLV8 struct {
name string
value string
actual any
expect any
noequal bool
}
func (test testTLV8) run(t *testing.T) {
if test.actual == nil {
return
}
src := &hap.Character{Value: test.value, Format: hap.FormatTLV8}
err := src.ReadTLV8(test.actual)
require.Nil(t, err)
require.Equal(t, test.expect, test.actual)
dst := &hap.Character{Format: hap.FormatTLV8}
err = dst.Write(test.actual)
require.Nil(t, err)
a, _ := base64.StdEncoding.DecodeString(test.value)
b, _ := base64.StdEncoding.DecodeString(dst.Value.(string))
t.Logf("%x\n", a)
t.Logf("%x\n", b)
if !test.noequal {
require.Equal(t, test.value, dst.Value)
}
}
func TestAqaraG3(t *testing.T) {
tests := []testTLV8{
{
name: "120",
value: "AQEA",
actual: &StreamingStatus{},
expect: &StreamingStatus{
Status: StreamingStatusAvailable,
},
},
{
name: "114",
value: "AaoBAQACEQEBAQIBAAAAAgECAwEABAEAAwsBAoAHAgI4BAMBHgAAAwsBAgAFAgLQAgMBHgAAAwsBAoACAgJoAQMBHgAAAwsBAuABAgIOAQMBHgAAAwsBAkABAgK0AAMBHgAAAwsBAgAFAgLAAwMBHgAAAwsBAgAEAgIAAwMBHgAAAwsBAoACAgLgAQMBHgAAAwsBAuABAgJoAQMBHgAAAwsBAkABAgLwAAMBHg==",
actual: &SupportedVideoStreamConfiguration{},
expect: &SupportedVideoStreamConfiguration{
Codecs: []VideoCodecConfiguration{
{
CodecType: VideoCodecTypeH264,
CodecParams: []VideoCodecParameters{
{
ProfileID: []byte{VideoCodecProfileMain},
Level: []byte{VideoCodecLevel31, VideoCodecLevel40},
CVOEnabled: []byte{0},
},
},
VideoAttrs: []VideoCodecAttributes{
{Width: 1920, Height: 1080, Framerate: 30},
{Width: 1280, Height: 720, Framerate: 30},
{Width: 640, Height: 360, Framerate: 30},
{Width: 480, Height: 270, Framerate: 30},
{Width: 320, Height: 180, Framerate: 30},
{Width: 1280, Height: 960, Framerate: 30},
{Width: 1024, Height: 768, Framerate: 30},
{Width: 640, Height: 480, Framerate: 30},
{Width: 480, Height: 360, Framerate: 30},
{Width: 320, Height: 240, Framerate: 30},
},
},
},
},
},
{
name: "115",
value: "AQ4BAQICCQEBAQIBAAMBAQIBAA==",
actual: &SupportedAudioStreamConfiguration{},
expect: &SupportedAudioStreamConfiguration{
Codecs: []AudioCodecConfiguration{
{
CodecType: AudioCodecTypeAACELD,
CodecParams: []AudioCodecParameters{
{
Channels: 1,
BitrateMode: AudioCodecBitrateVariable,
SampleRate: []byte{AudioCodecSampleRate16Khz},
},
},
},
},
ComfortNoiseSupport: 0,
},
},
{
name: "116",
value: "AgEAAAACAQEAAAIBAg==",
actual: &SupportedRTPConfiguration{},
expect: &SupportedRTPConfiguration{
SRTPCryptoType: []byte{CryptoAES_CM_128_HMAC_SHA1_80, CryptoAES_CM_256_HMAC_SHA1_80, CryptoDisabled},
},
},
}
for _, test := range tests {
t.Run(test.name, test.run)
}
}
func TestHomebridge(t *testing.T) {
tests := []testTLV8{
{
name: "114",
value: "AcUBAQACHQEBAAAAAQEBAAABAQICAQAAAAIBAQAAAgECAwEAAwsBAkABAgK0AAMBHgAAAwsBAkABAgLwAAMBDwAAAwsBAkABAgLwAAMBHgAAAwsBAuABAgIOAQMBHgAAAwsBAuABAgJoAQMBHgAAAwsBAoACAgJoAQMBHgAAAwsBAoACAgLgAQMBHgAAAwsBAgAFAgLQAgMBHgAAAwsBAgAFAgLAAwMBHgAAAwsBAoAHAgI4BAMBHgAAAwsBAkAGAgKwBAMBHg==",
actual: &SupportedVideoStreamConfiguration{},
expect: &SupportedVideoStreamConfiguration{
Codecs: []VideoCodecConfiguration{
{
CodecType: VideoCodecTypeH264,
CodecParams: []VideoCodecParameters{
{
ProfileID: []byte{VideoCodecProfileConstrainedBaseline, VideoCodecProfileMain, VideoCodecProfileHigh},
Level: []byte{VideoCodecLevel31, VideoCodecLevel32, VideoCodecLevel40},
},
},
VideoAttrs: []VideoCodecAttributes{
{Width: 320, Height: 180, Framerate: 30},
{Width: 320, Height: 240, Framerate: 15},
{Width: 320, Height: 240, Framerate: 30},
{Width: 480, Height: 270, Framerate: 30},
{Width: 480, Height: 360, Framerate: 30},
{Width: 640, Height: 360, Framerate: 30},
{Width: 640, Height: 480, Framerate: 30},
{Width: 1280, Height: 720, Framerate: 30},
{Width: 1280, Height: 960, Framerate: 30},
{Width: 1920, Height: 1080, Framerate: 30},
{Width: 1600, Height: 1200, Framerate: 30},
},
},
},
},
},
{
name: "116",
value: "AgEA",
actual: &SupportedRTPConfiguration{},
expect: &SupportedRTPConfiguration{
SRTPCryptoType: []byte{CryptoAES_CM_128_HMAC_SHA1_80},
},
},
}
for _, test := range tests {
t.Run(test.name, test.run)
}
}
func TestScrypted(t *testing.T) {
tests := []testTLV8{
{
name: "114",
value: "AVIBAQACEwEBAQIBAAAAAgEBAAACAQIDAQADCwECAA8CAnAIAwEeAAADCwECgAcCAjgEAwEeAAADCwECAAUCAtACAwEeAAADCwECQAECAvAAAwEP",
actual: &SupportedVideoStreamConfiguration{},
expect: &SupportedVideoStreamConfiguration{
Codecs: []VideoCodecConfiguration{
{
CodecType: VideoCodecTypeH264,
CodecParams: []VideoCodecParameters{
{
ProfileID: []byte{VideoCodecProfileMain},
Level: []byte{VideoCodecLevel31, VideoCodecLevel32, VideoCodecLevel40},
},
},
VideoAttrs: []VideoCodecAttributes{
{Width: 3840, Height: 2160, Framerate: 30},
{Width: 1920, Height: 1080, Framerate: 30},
{Width: 1280, Height: 720, Framerate: 30},
{Width: 320, Height: 240, Framerate: 15},
},
},
},
},
},
{
name: "115",
value: "AScBAQMCIgEBAQIBAAMBAAAAAwEAAAADAQEAAAMBAQAAAwECAAADAQICAQA=",
actual: &SupportedAudioStreamConfiguration{},
expect: &SupportedAudioStreamConfiguration{
Codecs: []AudioCodecConfiguration{
{
CodecType: AudioCodecTypeOpus,
CodecParams: []AudioCodecParameters{
{
Channels: 1,
BitrateMode: AudioCodecBitrateVariable,
SampleRate: []byte{
AudioCodecSampleRate8Khz, AudioCodecSampleRate8Khz,
AudioCodecSampleRate16Khz, AudioCodecSampleRate16Khz,
AudioCodecSampleRate24Khz, AudioCodecSampleRate24Khz,
},
},
},
},
},
ComfortNoiseSupport: 0,
},
},
{
name: "116",
value: "AgEAAAACAQI=",
actual: &SupportedRTPConfiguration{},
expect: &SupportedRTPConfiguration{
SRTPCryptoType: []byte{CryptoAES_CM_128_HMAC_SHA1_80, CryptoDisabled},
},
},
}
for _, test := range tests {
t.Run(test.name, test.run)
}
}
func TestHass(t *testing.T) {
tests := []testTLV8{
{
name: "114",
value: "AdABAQACFQMBAAEBAAEBAQEBAgIBAAIBAQIBAgMMAQJAAQICtAADAg8AAwwBAkABAgLwAAMCDwADDAECQAECArQAAwIeAAMMAQJAAQIC8AADAh4AAwwBAuABAgIOAQMCHgADDAEC4AECAmgBAwIeAAMMAQKAAgICaAEDAh4AAwwBAoACAgLgAQMCHgADDAECAAQCAkACAwIeAAMMAQIABAICAAMDAh4AAwwBAgAFAgLQAgMCHgADDAECAAUCAsADAwIeAAMMAQKABwICOAQDAh4A",
},
{
name: "115",
value: "AQ4BAQMCCQEBAQIBAAMBAgEOAQEDAgkBAQECAQADAQECAQA=",
},
}
for _, test := range tests {
t.Run(test.name, test.run)
}
}
@@ -0,0 +1,46 @@
package camera
const TypeSupportedVideoStreamConfiguration = "114"
type SupportedVideoStreamConfiguration struct {
Codecs []VideoCodecConfiguration `tlv8:"1"`
}
type VideoCodecConfiguration struct {
CodecType byte `tlv8:"1"`
CodecParams []VideoCodecParameters `tlv8:"2"`
VideoAttrs []VideoCodecAttributes `tlv8:"3"`
RTPParams []RTPParams `tlv8:"4"`
}
//goland:noinspection ALL
const (
VideoCodecTypeH264 = 0
VideoCodecProfileConstrainedBaseline = 0
VideoCodecProfileMain = 1
VideoCodecProfileHigh = 2
VideoCodecLevel31 = 0
VideoCodecLevel32 = 1
VideoCodecLevel40 = 2
VideoCodecPacketizationModeNonInterleaved = 0
VideoCodecCvoNotSuppported = 0
VideoCodecCvoSuppported = 1
)
type VideoCodecParameters struct {
ProfileID []byte `tlv8:"1"` // 0 - baseline, 1 - main, 2 - high
Level []byte `tlv8:"2"` // 0 - 3.1, 1 - 3.2, 2 - 4.0
PacketizationMode byte `tlv8:"3"` // only 0 - non interleaved
CVOEnabled []byte `tlv8:"4"` // 0 - not supported, 1 - supported
CVOID []byte `tlv8:"5"` // ID for CVO RTP extensio
}
type VideoCodecAttributes struct {
Width uint16 `tlv8:"1"`
Height uint16 `tlv8:"2"`
Framerate uint8 `tlv8:"3"`
}
@@ -0,0 +1,46 @@
package camera
const TypeSupportedAudioStreamConfiguration = "115"
type SupportedAudioStreamConfiguration struct {
Codecs []AudioCodecConfiguration `tlv8:"1"`
ComfortNoiseSupport byte `tlv8:"2"`
}
//goland:noinspection ALL
const (
AudioCodecTypePCMU = 0
AudioCodecTypePCMA = 1
AudioCodecTypeAACELD = 2
AudioCodecTypeOpus = 3
AudioCodecTypeMSBC = 4
AudioCodecTypeAMR = 5
AudioCodecTypeARMWB = 6
AudioCodecBitrateVariable = 0
AudioCodecBitrateConstant = 1
AudioCodecSampleRate8Khz = 0
AudioCodecSampleRate16Khz = 1
AudioCodecSampleRate24Khz = 2
RTPTimeAACELD8 = 60 // 8000/1000*60=480
RTPTimeAACELD16 = 30 // 16000/1000*30=480
RTPTimeAACELD24 = 20 // 24000/1000*20=480
RTPTimeAACLD16 = 60 // 16000/1000*60=960
RTPTimeAACLD24 = 40 // 24000/1000*40=960
)
type AudioCodecConfiguration struct {
CodecType byte `tlv8:"1"`
CodecParams []AudioCodecParameters `tlv8:"2"`
RTPParams []RTPParams `tlv8:"3"`
ComfortNoise []byte `tlv8:"4"`
}
type AudioCodecParameters struct {
Channels uint8 `tlv8:"1"`
BitrateMode byte `tlv8:"2"` // 0 - variable, 1 - constant
SampleRate []byte `tlv8:"3"` // 0 - 8000, 1 - 16000, 2 - 24000
RTPTime []uint8 `tlv8:"4"` // 20, 30, 40, 60
}
@@ -0,0 +1,14 @@
package camera
const TypeSupportedRTPConfiguration = "116"
//goland:noinspection ALL
const (
CryptoAES_CM_128_HMAC_SHA1_80 = 0
CryptoAES_CM_256_HMAC_SHA1_80 = 1
CryptoDisabled = 2
)
type SupportedRTPConfiguration struct {
SRTPCryptoType []byte `tlv8:"2"`
}
@@ -0,0 +1,32 @@
package camera
const TypeSelectedStreamConfiguration = "117"
type SelectedStreamConfiguration struct {
Control SessionControl `tlv8:"1"`
VideoCodec VideoCodecConfiguration `tlv8:"2"`
AudioCodec AudioCodecConfiguration `tlv8:"3"`
}
//goland:noinspection ALL
const (
SessionCommandEnd = 0
SessionCommandStart = 1
SessionCommandSuspend = 2
SessionCommandResume = 3
SessionCommandReconfigure = 4
)
type SessionControl struct {
SessionID string `tlv8:"1"`
Command byte `tlv8:"2"`
}
type RTPParams struct {
PayloadType uint8 `tlv8:"1"`
SSRC uint32 `tlv8:"2"`
MaxBitrate uint16 `tlv8:"3"`
RTCPInterval float32 `tlv8:"4"`
MaxMTU []uint16 `tlv8:"5"`
ComfortNoisePayloadType []uint8 `tlv8:"6"`
}
@@ -0,0 +1,33 @@
package camera
const TypeSetupEndpoints = "118"
type SetupEndpointsRequest struct {
SessionID string `tlv8:"1"`
Address Address `tlv8:"3"`
VideoCrypto SRTPCryptoSuite `tlv8:"4"`
AudioCrypto SRTPCryptoSuite `tlv8:"5"`
}
type SetupEndpointsResponse struct {
SessionID string `tlv8:"1"`
Status byte `tlv8:"2"`
Address Address `tlv8:"3"`
VideoCrypto SRTPCryptoSuite `tlv8:"4"`
AudioCrypto SRTPCryptoSuite `tlv8:"5"`
VideoSSRC uint32 `tlv8:"6"`
AudioSSRC uint32 `tlv8:"7"`
}
type Address struct {
IPVersion byte `tlv8:"1"`
IPAddr string `tlv8:"2"`
VideoRTPPort uint16 `tlv8:"3"`
AudioRTPPort uint16 `tlv8:"4"`
}
type SRTPCryptoSuite struct {
CryptoSuite byte `tlv8:"1"`
MasterKey string `tlv8:"2"` // 16 (AES_CM_128) or 32 (AES_256_CM)
MasterSalt string `tlv8:"3"` // 14 byte
}
@@ -0,0 +1,14 @@
package camera
const TypeStreamingStatus = "120"
type StreamingStatus struct {
Status byte `tlv8:"1"`
}
//goland:noinspection ALL
const (
StreamingStatusAvailable = 0
StreamingStatusInUse = 1
StreamingStatusUnavailable = 2
)
@@ -0,0 +1,11 @@
package camera
const TypeSupportedDataStreamTransportConfiguration = "130"
type SupportedDataStreamTransportConfiguration struct {
Configs []TransferTransportConfiguration `tlv8:"1"`
}
type TransferTransportConfiguration struct {
TransportType byte `tlv8:"1"`
}

Some files were not shown because too many files have changed in this diff Show More