Docs | Description&Proxy

**Streamedian** is a Javascript library which implements RTSP client for watching live streams in your browser that works directly on top of a standard HTML <video> element. It requires support of HTML5 Video with Media Sources Extensions for playback. Also player relies on server-side websocket proxy for retransmitting RTSP streams to browser.

It works by muxing RTP H264 and AAC payload into ISO BMFF (MP4) fragments.

**Streamedian** is written using ECMAScript 2015 standard

Browser support

Firefox v.42+
Chrome v.23+
OSX Safari v.8+
MS Edge v.13+
Opera v.15+
Android browser v.5.0+
IE Mobile v.11+

Not supported in iOS Safari and Internet Explorer.

Install from GitHub

Only player sources

npm install git://github.com/Streamedian/html5_rtsp_player.git

Browser side

Attach HTML Video with RTSP URL

<video id="test_video" controls autoplay src="rtsp://your_rtsp_stream/url"></video>

<video id="test_video" controls autoplay>
    <source src="rtsp://your_rtsp_stream/url">
</video>

You can ignore source by passing data-ignore="true":

<video id="test_video" controls autoplay>
    <source src="natively_supported_video_url" data-ignore="true">
    <source src="rtsp://your_rtsp_stream/url">
</video>

If browser can play that source, player will not be initialized for this element.

Setup player in your js:

import * as streamedian from 'streamedian/player';
// import WebsocketTransport from 'streamedian/transport/websocket';
// import RTSPClient from 'streamedian/client/rtsp/client';

let mediaElement = rtsp.attach(document.getElementById('test_video'));
let player = new streamedian.WSPlayer(mediaElement, options);

{
    // url: `${STREAM_URL}`,      // overrides mediaElement's sources
    modules: [
        {
            // client module constructor. Should be subclass or BaseClient. RTSPClient by default
            // client: RTSPClient,
            transport: {
               // client module constructor. Should be subclass or BaseTransport. WebsocketTransport by default
               // constructor: WebsocketTransport,
               options: {
                   // address of websocket proxy described below. ws${location.protocol=='https:'?'s':''}://${location.host}/ws/ by default
                   socket: "ws://websocket_proxy_address/ws"
               }
           }
        },
    ]
}

ES6 Modules support is required. You can use rollup to build this script:

rollup.config.js

import buble from 'rollup-plugin-buble';
import alias from 'rollup-plugin-alias';

const path = require('path');

export default {
    entry: path.join(__dirname, 'test.js'),
    targets: [
        {dest: path.join(__dirname, 'test.bundle.js'), format: 'umd'}
    ],
    sourceMap: true,
    plugins: [
        // uncomment if you want to transpile into es5
        //buble({
            //exclude: 'node_modules/**'
        //}),
        alias({
            bp_logger: path.join(__dirname,'node_modules/bp_logger/logger'),
            bp_event: path.join(__dirname,'node_modules/bp_event/event'),
            bp_statemachine: path.join(__dirname,'node_modules/bp_statemachine/statemachine'),
            wsp: path.join(__dirname,'node_modules/html5_rtsp_player/src')
        })
    ]

}

> npm install bp_event bp_logger bp_statemachine
> rollup -c

Include compiled script into your HTML:

<script src="streamedian.js"></script>

Server side

Sign up and install websocket proxy from http://streamedian.com
Register domain you want to use this player
Note that this package depends on systemd and gcc5+ runtime so it can be installed only on recent distribution versions.
Configure port and license file path in /etc/ws_rtsp.ini
This port should be open in your firewall. Also you can pass request to this port from your proxy. (for example: http://nginx.org/en/docs/http/websocket.html)
Run it
```
> service ws_rtsp start
```

Player architecture

Player comprises three main modules: transport, client and remuxer.

Transport responsible for data delivery and data queuing. It should fire connected, disconnected and data events. As soon data received, transport should push it into dataQueue and fire data event. Base class for transports can be found in core/base_transport. As a default, WebsocketTransport that relies on websocket proxy is implemented.

Client listens for data events, parse it into elementary stream packets and enqueue parsed data into it's own queue. Client can pass queued buffer to remuxer with samples event. To identify stream track client should fire tracks event. When ready, flush event can be used to construct media fragment and pass it to video presenter. Base class for transports can be found in core/base_client. Default client is RTSP client over websocket transport.

Remuxer prepares media fragment for video presenter and push it. There is only video presenter at the moment, based on media source extensions. Remuxer collects data into mp4 media fragments (moof+mdat) and pass it into source buffer.

Windows Installer

You install on your PC the restricted proxy server for streaming RTSP over websockets and the free RTSP Player.

We recommend to use the option "Service + NGINX web server" in the installer for testing of our software.
It allows to launch the RTSP Player in your default browser by double-click on the "Streamedian Player URL.lnk" file from your desktop.

Then you can set a RTSP link, for example: rtsp://<your's local PC ip address>:554/h264, and watch RTSP stream in your browser.
In text field you can type your own RTSP source url.

If you will change ws_rtsp.ini or wsp.lic files, be sure that you save in UTF8 encoding.

To watch stream inside local network need to change the ws url in html file

1. Need to turn off your firewall.
2. Find the string "127.0.0.1" in the "index.html" file and replace on an actual IP of PC, where the Proxy server was installed and launched.

Streamedian.player('test_video', {socket:"ws://<your's local PC ip address>:8080/ws/"});

The file is here:

C:\Users\Public\Documents\Streamedian\WS RTSP Player\nginx\html\index.html

Use in global network

Empty license file allows to watch the stream on your computer locally (intranet) only.
To stream into the Global network need to obtain the license file.
After registration on the website personal test key is gave in the personal cabinet.
To activate key, please, use the activation application tool that is placed:

C:\Program Files\Streamedian\WS RTSP Proxy Server\activation_app

After the start, select 'online' or 'offline' method of getting a license file

Online activation

In the case of online activation:
1. please log in to streamedian account, user:psw is the same as you registered on the website.
After successful login, you can see the list of available licenses and their parameters by selecting "Available licenses list"
2. To activate the key select point "Activate"
3. And Select the key from the list.
4. Enter your domain.
The license file will be uploaded to your server automatically and placed in the license proxy folder. The proxy will be restarting.
Please note that you can activate a key a limited number of times. The number of remaining activation attempts can be seen when choosing a license.

Offline activation

When you select offline activation the 'machine_info.bin' will be created here:

C:\Users\Public\Documents\Streamedian\WS RTSP Proxy Server\machine_info.bin

1. Go to the personal cabinet(www.stremedian.com/cabinet)
2. Choose "Offline activation" tab
3. Select the key which you will activate
4. Upload 'machine_info.bin'.
5. Enter domain name or IP of player host
6. Click "Activate" button
7. Download and copy the wsp.lic file here:

C:\Users\Public\Documents\Streamedian\WS RTSP Proxy Server\wsp.lic

8. To restart the service run:

C:\Program Files\Streamedian\WS RTSP Proxy Server\restart_service.bat

If you want to change the limit on the number of channels or domains, you can purchase a license.

How to watch the logs of proxy

1. Uninstall the previous version of the Streamedian WS RTSP Proxy Server.

2. Download DebugView.zip. Unpack it and launch DbgView.exe. You will need to accept the License Agreement. The DebugView is an application of a wholly owned subsidiary of Microsoft Corporation.

3. Install the new Streamedian WS RTSP Proxy Server.

4. Reproduce an issue.

5. Watch logs on DebugView

Mac Installer

You install on your PC the restricted proxy server for streaming RTSP over websockets and the free RTSP Player.

We recommend to use the option "NGINX web server" in the installer for testing of our software.
It allows to launch the RTSP Player in your default browser by double-click on the "LauchPlayer" from "/Applications/Streamedian WS RTSP Proxy Server/". Or type "http://127.0.0.1:8088/" in your browser.

Then you can set a RTSP link, for example: rtsp://<your's local PC ip address>:554/h264, and watch RTSP stream in your browser.
In text field you can type your own RTSP source url.

The configuration file is placed here:

/Users/Shared/Streamedian/WS RTSP Proxy Server/ws_rtsp.ini

If you will change ws_rtsp.ini or wsp.lic files, be sure that you save in UTF8 encoding.

To watch stream inside local network need to change the ws url in html file

1. Need to turn off your firewall.
2. Find the string "127.0.0.1" in the "index.html" file and replace on an actual IP of PC, where the Proxy server was installed and launched.

Streamedian.player('test_video', {socket:"ws://<your's local PC ip address>:8080/ws/"});

The file is here:

/Library/Application Support/Streamedian/WS RTSP Proxy Server/index.html

Use in global network

/Library/Application Support/Streamedian/WS RTSP Proxy Server/activation_app

After the start, select 'online' or 'offline' method of getting a license file

Online activation

Offline activation

When you select offline activation the 'machine_info.bin' will be created here:

/Users/Shared/Streamedian/WS RTSP Proxy Server/machine_info.bin

/Users/Shared/Streamedian/WS RTSP Proxy Server/wsp.lic

8. To restart the service run:

/Library/Application Support/Streamedian/WS RTSP Proxy Server/restart_service.sh

If you want to change the limit on the number of channels or domains, you can purchase a license.

Ubuntu/Debian/Raspberry Installer

ou install on your PC the restricted proxy server for streaming RTSP over websockets and the free RTSP Player. The current package installs configuration files for the NGINX and the Apache web servers.

We recommend to install the NGINX web server for most easy testing of our software. To install the NGINX web server, please type in the Terminal:

sudo apt install nginx

It allows to launch and test the RTSP Player in your default browser by typing "localhost:8088" (without quotes).

Then you can set a RTSP link, for example: rtsp://<your's local PC ip address>:554/h264, and watch RTSP stream in your browser. In text field you can type your own RTSP source url.

The configuration file is placed here:

/etc/ws_rtsp.ini

If you will change ws_rtsp.ini or wsp.lic files, be sure that you save in UTF8 encoding.

If you would like to use the Apache web server for testing, just type in your browser "localhost:8088" (without quotes).
If you have installed the Apache web server after this package, please type in the Terminal:

sudo a2enconf ws_rtsp_proxy_stream.conf
sudo service apache2 reload

To watch stream inside local network need to change the ws url in html file

1. Need to turn off your firewall.
2. Find the string "127.0.0.1" in the "index.html" file and replace on an actual IP of PC, where the Proxy server was installed and launched.

Streamedian.player('test_video', {socket:"ws://<your's local PC ip address>:8080/ws/"});

The file is here:

/var/www/ws_rtsp_proxy/index.html

Use in global network

/usr/bin/wsp/activation_app

After the start, select 'online' or 'offline' method of getting a license file

Online activation

Offline activation

When you select offline activation the 'machine_info.bin' will be created here:

/usr/share/wsp/machine_info.bin

/usr/share/wsp/wsp.lic

8. To restart the service run:

/usr/share/wsp/restart_service.sh

If you want to change the limit on the number of channels or domains, you can purchase a license.

systemctl start ws_rtsp

Fedora/CentOS Installer

You install on your PC the restricted proxy server for streaming RTSP over websockets and the free RTSP Player.

If you would like to test our software in the simplest way, then we advice you to use the following way:

1. Run the script /usr/share/wsp/nginx/nginx_setup.sh.
It will install the NGINX web server and setup your environment.

2. Open a web browser on some PC in your local subnet.
And type "IP_ADDRESS:8088", where 'IP_ADDRESS' is the IP of PC, where the proxy server was installed.

3. Then you can set a RTSP link, for example: rtsp://<your's local PC ip address>:554/h264,
and watch a RTSP stream in your browser. In the text field you can type your own RTSP source url.

The configuration file is placed here:

/etc/ws_rtsp.ini

If you will change ws_rtsp.ini or wsp.lic files, be sure that you save in UTF8 encoding.

sudo a2enconf ws_rtsp_proxy_stream.conf
sudo service apache2 reload

To watch stream inside local network need to change the ws url in html file

1. Need to turn off your firewall.
2. Find the string "127.0.0.1" in the "index.html" file and replace on an actual IP of PC, where the Proxy server was installed and launched.

Streamedian.player('test_video', {socket:"ws://<your's local PC ip address>:8080/ws/"});

The file is here:

/var/www/ws_rtsp_proxy/index.html

Use in global network

/usr/bin/wsp/activation_app

After the start, select 'online' or 'offline' method of getting a license file

Online activation

Offline activation

When you select offline activation the 'machine_info.bin' will be created here:

/usr/share/wsp/machine_info.bin

/usr/share/wsp/wsp.lic

8. To restart the service run:

/usr/share/wsp/restart_service.sh

If you want to change the limit on the number of channels or domains, you can purchase a license.

How RTSP proxy works?

RTSP player establish connection with proxy with following protocol:

Connect to RTSP channel by connecting websocket with "rtsp" protocol specified and get connection id

c>s:
WSP/1.1 INIT\r\n
seq: <sequence_id for response tracking>
host: <RTSP stream host>\r\n
port: <RTSP stream port>\r\n\r\n

s>c:
WSP/1.1 200 OK\r\n
seq: <sequence_id for response tracking>
channel: <channel_id>\r\n\r\n

Error codes >= 400 means error

Connect to RTP channel by connecting websocket with "rtp" protocol

c>s:
WSP/1.1 JOIN\r\n
seq: <sequence_id for response tracking>\r\n
channel: <channel_id achieved from RTSP socket initialization>\r\n\r\n

s>c:
WSP/1.1 200 OK\r\n
seq: <sequence_id for response tracking>\r\n\r\n

Error codes >= 400 means error

Send RTSP commands wrapped into WSP protocol:

c>s:
WSP/1.1 WRAP\r\n
seq: <sequence_id for response tracking>\r\n
\r\n
<RTSP_PROTOCOL_DATA>

s>c:
WSP/1.1 200 OK\r\n
channel: <channel_id>\r\n
\r\n
<RTSP_PROTOCOL_RESPONSE>

Error codes >= 400 means error

RTP channel should send interleaved data with 4 byte header ($<channel> <size>). Separate RTP is not supported at this moment

RTSP WS Tunnel Client (supported since version 1.8.4)

For whom this solution will be convenient.
- Who has video cameras in the local network, but there is no global IP address to access the cameras from the global network.
- Who does not have the opportunity to open the necessary ports on routers and firewalls hidden behind routers.
- If the company has a remote warehouse or several offices and production premises located in different places of the city. The ability to encrypt the channel (wss tunnel) will allow scramble your cameras.

WebSocket RTSP Tunnel Client can be installed:
- to any computer on the local network,
- to one of the most popular boards, for example, RasperiPi or OrangePi,
- flash directly into your router (using an alternative firmware).

Distribution scheme for data

Configuration for Nginx

server {
    listen 80;
    server_name <domain name>;

    location / {
	root /srv/proxy/;
	index index.html;
    }

    location = /ws/ {
	proxy_pass http://localhost:8080/ws/;
    }

    location = /tunnel/ {
	proxy_pass http://localhost:4000/tunnel/;
    }
}

libssl

For running in raspberry may need library "libssl1.0.0" or 1.0.1 To install it insert this line in "/etc/apt/sources.list":

deb http://archive.raspbian.org/raspbian stretch main contrib non-free

Then run following command in respberry terminal:

sudo apt update
sudo apt install libssl1.0.0

Controll

Commands to control the tunnel client service

systemctl start rtsp_ws_tunnel_client
systemctl stop rtsp_ws_tunnel_client

Config file /etc/rtsp_ws_tunnel_client.ini

before work need setup config parameters /etc/rtsp_ws_tunnel_client.ini

systemctl stop rtsp_ws_tunnel_client

[General]
//RTSP WS tunnel client client id
client_id=RTSP WS tunnel client

//websocket proxy access token - should be the same as on your proxy side
proxy_token=OWJlN2FiMDMtZGEyMC00NGM1LTg5ZDItOGVlMjM

// websocket proxy address - should look to your ws://
//if wss:// - set up "ssl=on" below
proxy_url=ws://192.168.1.180:8088/tunnel/

// RTSP WS tunnel client plugin dir
plugin_path=/usr/lib/rwtc/

// wss connection off - for ws:// | on - for wss://
ssl=off

[CAMERAS] - list your rtsp sources - should be TCP type
url=rtsp://192.168.1.185:554/h264 example rtsp url
url=rtsp://192.168.1.186:554/h264 example rtsp url
url=rtsp://192.168.1.187:554/h264 example rtsp url
url=rtsp://192.168.1.188:554/h264 example rtsp url
url=rtsp://192.168.1.189:554/h264 example rtsp url

you will see this list on the test page when the page makes the first connection to the proxy after changing the ini file need restart tunnel client service

systemctl start rtsp_ws_tunnel_client

Machine Learning object recognition (ml5.js library integration)

Together implementation object recognition on video using ml5.js and HTML5 RTSP player. ml5.js is a library that makes machine learning more accessible and performs all image procesing in a browser side. We integrated it with player in the demo PLAYER FREE & ml5.js ML with the Yolo model to detect objects in the video.

ml5.js integration based on HTML5 RTSP player test page

p5 and ml5 libraries loading

<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/p5.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/addons/p5.dom.min.js"></script>
<script src="https://unpkg.com/ml5@0.4.3/dist/ml5.min.js"></script>

Canvas adding to test page for bounding boxes displaying

<div style="position: relative;">
    <video id="test_video" width="720" height="360" muted autoplay style="position: absolute;"></video>
    <canvas id="test_canvas" style="position: absolute;"></canvas>
</div>

Javascript code for object detection and bounding bosxes drawing

let yolo;
let ctx;
let width;
let height;

function initObjectsDetection(playerId, canvasId, rtspLink) {
    const updateTime = 3000; // Interval in millis between image capture for objects detection
    const video = document.getElementById(playerId);
    const canvas = document.getElementById(canvasId);
    ctx = canvas.getContext('2d');
    width = video.offsetWidth;
    height = video.offsetHeight;

    // Resize canvas
    canvas.width = width;
    canvas.height = height;

    // Load Yolo model
    const ml = new ml5.YOLO(video);
    ml.loadModel().then(function(model) {
        yolo = model;

        // Start RTSP link playback
        setPlayerSource(rtspLink);

        // Set interval for objects detection
        setInterval(objectDetect, updateTime);
    });
}

function objectDetect() {
    yolo.detect(function(err, results) {
        if (results !== undefined) {
            // Draw results on canvas
            draw(results);
        }
    });
}

function draw(results) {
    ctx.clearRect(0, 0, width, height);
    ctx.font = "24px helvetica";
    ctx.textBaseline = "top";

    for (let i = 0; i < results.length; i++) { //Iterating through all results
        // Bounding box drawing
        ctx.strokeStyle = "#2fff00";
        ctx.lineWidth = 1;
        ctx.strokeRect(results[i].x * width, results[i].y * height, results[i].w * width, results[i].h * height);


        // Text drawing last to ensure it's on top
        const textWidth = ctx.measureText(results[i].label).width;
        const textHeight = parseInt("24px helvetica", 10);
        ctx.fillStyle = "#000000";
        ctx.fillText(results[i].label, results[i].x * width, results[i].y * height - 5);
    }
}

Object detection initialization calling after streamedian player initialized

var player = Streamedian.player('test_video', playerOptions);
var nativePlayer = document.getElementById('test_video');
var statusText = document.getElementById('statusText');

initObjectsDetection("test_video", "test_canvas", 'rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov');

Demo: PLAYER FREE & ml5.js ML

More about ml5.js