Play fmp4 Video Stream over WebSocket

For all installation solutions, refer to Solutions Installation

Implementation Principle

Receive data from WebSocket
Create MediaSource instance
Create SourceBuffer instance
Pass data to SourceBuffer instance
Pass MediaSource instance to video tag
Draw video to canvas

Install as Standalone Hooks

If using via hooks, execute the following installation:

npmyarnpnpm

shell

npm install @havue/use-ws-video --save

shell

yarn add @havue/use-ws-video

shell

pnpm install @havue/use-ws-video

Install as Standalone JavaScript Class

If not using hooks:

npmyarnpnpm

shell

npm install @havue/ws-video-manager --save

shell

yarn add @havue/ws-video-manager

shell

pnpm install @havue/ws-video-manager

Usage

Import useVideoPlay hook

import { useWsVideo } from 'havue'
// or
import { useWsVideo } from '@havue/hooks'
// or
import { useWsVideo } from '@havue/use-ws-video'

Import JavaScript manager class

import { WsVideoManager } from 'havue'
// or
import { WsVideoManager } from '@havue/solutions'
// or
import { WsVideoManager } from '@havue/ws-video-manager'

Function Declaration

import type { Ref, MaybeRef } from 'vue'
import type { RenderConstructorOptionType, VideoInfo, WsVideoManager } from '@havue/ws-video-manager'

export type UseWsVideoCanvasResizeOption = {
  /** 是否启用自动更新canvas width 和 height属性，默认为true */
  enable?: boolean
  /** 设置canvas width 和 height时，
   * 缩放的比例，即元素实际尺寸乘以scale，
   * 放大是为了画面更清晰
   * 默认 1
   */
  scale?: number
  /** 限制canvas width最大值，默认1920 */
  maxWidth?: number
  /** 限制canvas height最大值，默认1080 */
  maxHeight?: number
}

export type UseWsVideoParamsOptions = {
  /** websocket 地址 */
  wsUrl: MaybeRef<string | undefined>
  /** 是否播放视频 */
  isReady: MaybeRef<boolean>
  /** 使用的WsVideoManager 实例 默认为wsVideoPlayer */
  wsVideoPlayerIns?: WsVideoManager
  /** 视频渲染到的canvas元素, 不传会返回一个元素引用变量：canvasRef */
  target?: MaybeRef<HTMLCanvasElement | undefined>
  /** 是否自动更新canvas width和height属性的配置， 默认为 USE_WS_VIDEO_DEFAULT_RESIZE_OPTIONS */
  canvasResize?: MaybeRef<UseWsVideoCanvasResizeOption | undefined>
  /** 视口中元素不可见时断开连接， 默认为true */
  closeOnHidden?: MaybeRef<boolean>
  /** 自定义Render配置 */
  renderOptions?: MaybeRef<Partial<RenderConstructorOptionType>>
}

// canvasResize 默认值
export const USE_WS_VIDEO_DEFAULT_RESIZE_OPTIONS = Object.freeze({
  enable: true,
  scale: 1,
  maxWidth: 1920,
  maxHeight: 1080
})

export type UseWsVideoReturnType = {
  /** canvas引用 */
  canvasRef: Ref<HTMLCanvasElement | undefined>
  /** 是否静音 */
  isMuted: Ref<boolean>
  /** 是否暂停 */
  isPaused: Ref<boolean>
  /** 视频信息 */
  videoInfo: Ref<VideoInfo>
  /** 已经连接的WebSocket地址列表 */
  linkedWsUrlList: Ref<string[]>
  /** 视频流地址是否已添加 */
  isLinked: Ref<boolean>
  /** 是否达到websocket拉流数最大值 */
  isReachConnectLimit: Ref<boolean>
  /** 暂停其他WebSocket视频流的音频播放 */
  pauseOtherAudio: () => void
  /** 设置当前WebSocket视频流的音频是否暂停 */
  setAudioMutedState: (muted: boolean) => void
  /** 暂停其他WebSocket视频流的视频播放 */
  pauseOtherVideo: () => void
  /** 设置当前WebSocket视频流的视频是否暂停 */
  setOneVideoPausedState: (paused: boolean) => void
  /** 设置所有WebSocket视频流的视频是否暂停 */
  setAllVideoPausedState: (paused: boolean) => void
  /** 刷新当前WebSocket视频流的时间，如果连接断开会进行重连 */
  refresh: () => void
}

/**
 * WebSocket video stream playback
 * @param {UseWsVideoParamsOptions } options Configuration options
 * @returns 
 */
export function useVideoPlay(options: UseWsVideoParamsOptions ) : UseWsVideoReturnType

TIP

If canvas appears blurry, recommend setting options.canvasResize to increase canvas width/height values, e.g. scale by window.devicePixelRatio

useVideoPlay({
  canvasResize: {
    enable: true,
    scale: window.devicePixelRatio || 1,
  }
})

Example

websocket url:

width:

height:

Click to view code

templatescriptstyle

vue

<template>
  <div>
    <div class="form-box">
      <div class="form-item"><span class="label">websocket url:</span><input v-model="url" /></div>
      <div class="form-item"><span class="label">width:</span><input v-model="width" /></div>
      <div class="form-item"><span class="label">height:</span><input v-model="height" /></div>
    </div>
    <div class="video-player" :style="{ width: `${width}px`, height: `${height}px` }">
      <canvas ref="canvasRef" :width="width" :height="height"></canvas>
    </div>
  </div>
</template>

<script setup lang="ts">
import { ref } from 'vue'
// import { useWsVideo } from '@havue/use-ws-video'
import { useWsVideo } from '@havue/hooks'

const url = ref('')
const width = ref(640)
const height = ref(320)

const canvasRef = ref()

useWsVideo({
  wsUrl: url,
  isReady: true,
  target: canvasRef
})
</script>

scss

<style lang="scss" scoped>
.form-box {
  padding: 15px;
  background: #25465845;

  .form-item {
    margin-bottom: 5px;

    .label {
      display: inline-block;
      width: 120px;
      margin-right: 20px;
      text-align: right;
    }

    input {
      padding: 1px 8px;
      border: 1px solid gray;
    }
  }
}

.video-player {
  background: #578895;

  canvas {
    width: 100%;
    height: 100%;
  }
}
</style>

useWsVideo Configuration Options

Parameter	Description	Type	Default
wsUrl	WebSocket URL	`string`	-
isReady	Whether to play	`boolean \| Ref<boolean>`	-
wsVideoPlayerIns	WsVideoManager instance	`WsVideoManager`	`WsVideoManager()`
target	Canvas element (auto creates ref if not provided)	`HTMLCanvasElement \| Ref<HTMLCanvasElement>`	-
autoResizeCanvas	Auto-resize canvas dimensions	`CanvasResizeOption \| Ref<CanvasResizeOption>`	`false`
closeOnHidden	Disconnect when element not visible	`boolean`	`true`
renderOptions	Render class configuration	`Partial<RenderConstructorOptionType> \| undefined`	`undefined`

WsVideoManager

Constructor

Click to view code

loaderrender

/** 心跳配置 */
type HeartbeatConfigType = {
  /** 只发送一次 */
  once: boolean
  /** 心跳消息 */
  message: string
  /** 时间间隔 */
  interval?: number
}

/** 重连配置 */
type InterruptConfigType = {
  /** 是否重连 */
  reconnect: boolean
  /** 最大重连次数 */
  maxReconnectTimes: number
  /** 每次重连延时 */
  delay: number
}

export type WebSocketOptionsType = {
  /** WebSocket 子协议 WebSocket(url: string, protocols: string | string[]) */
  protocols?: string | string[]
  /** WebSocket 连接所传输二进制数据的类型 */
  binaryType?: WebSocket['binaryType']
  heartbeat?: HeartbeatConfigType
  interrupt?: InterruptConfigType
}

export type RenderConstructorOptionType = {
  /** 当前播放currentTime和最新视频时长最多相差 秒数，默认0.3s */
  liveMaxLatency: number
  /** 最多缓存ws传输的未处理的buffer数据大小, 默认200kb */
  maxCacheBufByte: number
  /** 最多存储的时间，用于清除在currentTime之前x秒时间节点前的buffer数据, 默认10s */
  maxCache: number
}

export const WS_VIDEO_RENDER_DEFAULT_OPTIONS = Object.freeze({
  liveMaxLatency: 0.3,
  maxCacheBufByte: 200 * 1024,
  maxCache: 10
})

export enum AudioState {
  NOTMUTED = 'notmuted',
  MUTED = 'muted'
}

export enum VideoState {
  PLAY = 'play',
  PAUSE = 'pause'
}

export type VideoInfo = {
  width: number
  height: number
}

export enum RenderEventsEnum {
  AUDIO_STATE_CHANGE = 'audioStateChange',
  VIDEO_STATE_CHANGE = 'videoStateChange',
  VIDEO_INFO_UPDATE = 'videoInfoUpdate'
}

export type RenderEvents = {
  [RenderEventsEnum.AUDIO_STATE_CHANGE]: (s: AudioState) => void
  [RenderEventsEnum.VIDEO_STATE_CHANGE]: (s: VideoState) => void
  [RenderEventsEnum.VIDEO_INFO_UPDATE]: (info: VideoInfo) => void
}

import type { WebSocketOptionsType } from '../loader/websocket-loader'
import type { RenderConstructorOptionType, VideoInfo } from '../render'

export type WsVideoManaCstorOptionType = {
  /** 预监流连接数量限制, 移动端默认10个，pc端默认32个 */
  connectLimit?: number
  /** WebSocketLoader 实例配置 */
  wsOptions?: WebSocketOptionsType
  /**
   * websocket重连时，重新解析视频编码方式，
   * 默认 true
   */
  reparseMimeOnReconnect?: boolean
  /** Render 实例配置 */
  renderOptions?: Partial<RenderConstructorOptionType>
  /**
   * 是否使用WebGL，
   * 默认 false，
   * WebGL在不同游览器，以及受限于显存，不能同时创建过多WebGL上下文，一般8-16个 */
  useWebgl?: boolean
}

const DEFAULT_OPTIONS: Required<WsVideoManaCstorOptionType> = Object.freeze({
  connectLimit: isMobile ? 10 : 32,
  wsOptions: {
    binaryType: 'arraybuffer' as WebSocket['binaryType']
  },
  reparseMimeOnReconnect: true,
  renderOptions: RENDER_DEFAULT_OPTIONS,
  useWebgl: false
})

type WsInfoType = {
  /** 需要绘制的canvas列表 */
  canvasMap: Map<HTMLCanvasElement, CanvasDrawer>
  /** WebSocketLoader 实例 */
  socket: WebSocketLoader
  /** socket连接渲染render实例 */
  render: Render
}

export enum EventEnums {
  WS_URL_CHANGE = 'wsUrlChange',
  SOCKET_CLOSE = 'socketClose',
  CONNECT_LIMIT = 'connectLimit'
}

type Events = {
  [EventEnums.WS_URL_CHANGE]: (urls: string[]) => void
  [RenderEventsEnum.AUDIO_STATE_CHANGE]: (url: string, state: AudioState) => void
  [RenderEventsEnum.VIDEO_INFO_UPDATE]: (url: string, info: VideoInfo) => void
  [RenderEventsEnum.VIDEO_STATE_CHANGE]: (url: string, state: VideoState) => void
  [EventEnums.SOCKET_CLOSE]: (url: string) => void
  [EventEnums.CONNECT_LIMIT]: () => void
}

export const WsVideoManagerEventEnums = Object.assign({}, EventEnums, RenderEventsEnum)

export class WsVideoManager extends EventBus<Events> {
  constructor(options?: WsVideoManaCstorOptionType): void
}

Instance Properties

Property	Description	Type
linkedUrlList	Connected WebSocket URLs	`string[]`
connectLimit	Max allowed WebSocket connections	`number`
addCanvas	Add WebSocket URL and canvas	`(canvas: HTMLCanvasElement, url: string, renderOptions?: Partial<RenderConstructorOptionType>) => void`
removeCanvas	Remove canvas	`(canvas: HTMLCanvasElement) => void`
isCanvasExist	Check canvas existence	`(canvas: HTMLCanvasElement) => boolean`
updateRenderOptions	Update render configuration	`(url: string, options?: Partial<RenderConstructorOptionType>) => void`
setAllVideoMutedState	Set mute state for all videos	`(muted: boolean) => void`
setOneMutedState	Set mute state for single video	`(url: string, muted: boolean) => void`
getOneMutedState	Get mute state for single video	`(url: string) => void`
playOneAudio	Play audio from single video only	`(url: string) => void`
setAllVideoPausedState	Set pause state for all videos	`(paused: boolean) => void`
setOneVideoPausedState	Set pause state for single video	`(url: string, paused: boolean) => void`
getOneVideoPausedState	Get pause state for single video	`(url: string) => void`
playOneVideo	Play single video only	`(url: string) => void`
refresh	Refresh video playback (reconnect if needed)	`(url?: string) => void`
on	Event listener	`(event: string, cb: (...args) => void) => void`
off	Remove event listener	`(event: string, cb?: (...args) => void) => void`
emit	Trigger event	`(event: string, ...args) => void`
destroy	Destroy instance	`() => void`

Events

Event	Description	Type
`wsUrlChange`	WebSocket URL list updated	`(urls: string[]) => void`
`audioStateChange`	Audio mute state changed	`(url: string, state: AudioState) => void`
`videoStateChange`	Video playback state changed	`(url: string, state: VideoState) => void`
`videoInfoUpdate`	Video dimension info updated	`(url: string, state: VideoInfo) => void`
`socketClose`	WebSocket connection closed	`(url: string) => void`
`connectLimit`	Exceeded connection limit	`() => void`

export enum AudioState {
  NOTMUTED = 'notmuted',
  MUTED = 'muted'
}

export enum VideoState {
  PLAY = 'play',
  PAUSE = 'pause'
}

export type VideoInfo = {
  width: number
  height: number
}

Play fmp4 Video Stream over WebSocket ​

Implementation Principle ​

Install as Standalone Hooks ​

Install as Standalone JavaScript Class ​

Usage ​

Function Declaration ​

Example ​

useWsVideo Configuration Options ​

WsVideoManager ​

Constructor ​

Instance Properties ​

Events ​

Play fmp4 Video Stream over WebSocket

Implementation Principle

Install as Standalone Hooks

Install as Standalone JavaScript Class

Usage

Function Declaration

Example

useWsVideo Configuration Options

WsVideoManager

Constructor

Instance Properties

Events