Video Portal for video transcoding and audio transcribing
Project Overview:
YouTube-like video portal for video transcoding and audio transcribing
Purpose:
Proof of Concept (PoC) for enhancing the demo of VOD and video processor
Key Technologies:
Django (Python3), PostgreSQL, Object Storage API, Kubernetes APIs
Scope of the project
Building a Minimum Viable Product (MVP) video portal. The project is broken down into multiple phases and delivered with PoCs weekly.
The phases with weekly PoC demos:
- PoC1: Django and PostgreSQL for SQL queries.
- PoC2: Django and Kubernetes API integration with RBAC credentials.
- PoC3: HTML form with pre-signed URL for uploading files to Object storage (MinIO).
- PoC4: Kubernetes API to trigger video processing.
- PoC5: Implementing a Video Portal with thumbnail and preview functionality.
- PoC6: Retrieving files from MinIO bucket to VOD pod on Kubernetes via Django.
Each PoC demo is delivered as a container image and deployed on the Kubernetes platform.
PoC1: Django and PostgreSQL for SQL queries.
- Design and test SQL queries of database for transcoding video jobs
PoC2: Django and Kubernetes API integration with RBAC credentials.
- Use RBAC( Role-Based Access Control) credential for Kubernetes workload requests, status, and logs
- Integration with Kubernetes API for workload management
- Invoke Kubernetes API to create a job
Experimented 4 with Pythons scripts:
- Use a YAML file to apply “resource” to Kubernetes
- yaml_file = ‘/Users/lgu/Documents/PycharmProjects/pythonProject/job2.yml’
- utils.create_from_yaml(aApiClient,yaml_file,verbose=True,namespace=”poc2test”)
- Use V1 API to apply k8s Job crud
- api_response = api_instance.create_namespaced_job(body=job,namespace=”poc2test”)
- api_response = api_instance.read_namespaced_job_status(name=..,namespace=..)
- api_response = api_instance.patch_namespaced_job(…)
- api_instance.delete_namespaced_job(…)
- Use V1 API to apply k8s deployment
- k8s_apps_v1.create_namespaced_deployment(body=dep, namespace=”poc2test”)
- Show the log in a Kubernetes Pod
- api_instance.read_namespaced_pod_status(name, namespace)
PoC3: HTML form with pre-signed URL for uploading files to Object storage (MinIO).
- Upload video files to MinIO using Django backend with MinIO API
- Create an HTML form, get the MinIO bucket list, and return the upload status
- Get a pre-signed URL for uploading files
PoC4: Kubernetes API to trigger video processing.
- Get a pre-signed URL for video file in MinIO source bucket
- Kubernetes API to trigger a Job for video processing (YAML)
- Video Processor container (3 tasks):
-
- Transcoding Video in h.264
-
- Transcoding Video in h.265
-
- Transcribing Audio into text (in WebVTT)
- Real-time response of a K8s Pod. Use StreamingHttpResponse API, and JavaScript code to create an EventSource object from the Django template
PoC5: Implementing a Video Portal with thumbnail and preview functionality.
- Implement a Web Portal to present a list of processed videos in a YouTube-like UI
- Enhance the audience experience with JavaScript (Thumbnail and animated preview GIF)
- A HTML5 video player supporting both HLS and DASH video streaming
PoC6: Retrieving files from MinIO bucket to VOD pod on Kubernetes via Django.
Retrieve video from MinIO bucket and transfer to VOD Pod in Kubernetes (use MinIO API and K8s API for transferring files) Provide content to VOD Pod for VOD streaming
Final: Integrate all PoCs into a Video Portal
Upload video to Object bucket and submit a video processor pod to transcode video and transcribe audio for subtitle