Live-Video-Stream App
Tags: python, socket, opencv
Published: 2021-06-16 18:33:19

Hello everyone, 

In this blog we are going to discuss how we can create Live Video Stream app .
Before we start with the demo part first lets discuss the basic terminologies that will help in understanding the blog better.


## What is OpenCV?

OpenCV is a library that works with almost all programming languages therefor known as cross-platform library which is mainly used for Image Processing that is used in Computer Vision

## What is Socket Programming?

Socket programming is a way to connect two nodes on a network to communicate with each other. [for details](https://www.geeksforgeeks.org/socket-programming-cc/)


## What is socket?

A socket is one endpoint of a two way communication link between two programs running on the network.


## What are the Network Protocol ?

Protocol are the set of rules that are followed while transmitting data over network.
Here we are going to use TCP( Transmission Control Protocol).

## what is TCP?

TCP Stands for transmission control protocol that enables application programs and computing devices to exchange messages over a network, it is designed to send packets across the internet and ensure the delivery of the data. [For Detail](https://www.fortinet.com/resources/cyberglossary/tcp-ip)

![Socket](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l5ebor9odn3z2bq5z28b.jpg)

## Prerequisites:

- Numpy 
- OpenCv

- Now lets begin with the code

first we need to create a Server program, Client connects to the server program.

```python
soc = socket.socket()
soc.bind(("", 5678))
soc.listen()
session, address = soc.accept() from anywhere
print(session.recv(2046)) 
camIndex = 'http://IP-of-WebCam:8080/video'
cam = cv.VideoCapture(camIndex) 

def sender():
    while True:
        ret, photo = cam.read()
        photo = cv.resize(photo, (640, 480))
        print(photo.shape)
        if ret:
            session.send(np.ndarray.tobytes(photo))
        else: print("No Frame Found")

def receiver():
    framesLost = 0
    print("Connecton Established")
    while True:
        framesLost += 1 
        data = session.recv(100000000)
        if(data == b'finished'):
            print("Finished")
            cam.release()
            session.close()
        else:
            photo =  np.frombuffer(data, dtype=np.uint8)
            if len(photo) == 640*480*3:
                cv.imshow('Hi I am Client', photo.reshape(480, 640, 3))
                if cv.waitKey(1) == 13:
                    session.send(b'finished')
                    cam.release()
                    cv.destroyAllWindows()
                    break
            else:
                print("Lost {} frames".format(framesLost) )
                
threading.Thread(target=sender).start()
threading.Thread(target=receiver).start()
```
now let's understand the code 

```python
soc = socket.socket()
soc.bind(("", 5678)) 
```
Here **socket** is a function that comes from the socket library, to use it you need to import the socket library first : `import socket` 

Here **bind** function is use to bind the IP to the port number. In the above code  **" "** means the IP of local system and **5678** is the port number that we have bind with the local host IP.

 ```python
session, address = soc.accept() #here it will accepts the request from anywhere
print(session.recv(2046)) 
camIndex = 'http://IP_of_IPWebCam:8080/video' # here this IP is of IPCAMERA app 
cam = cv.VideoCapture(camIndex)
``` 
Here **accept()** is use to accept the the request from the client.

**session.recv()** show the receives data on the socket and store it in a buffer and here **2046** is the size of buffer in bytes.

**camIndex** variable stores the address of my camera, here I am using my mobile camera, for that I have used the **IP WebCam** android app which create a server through which we can use the camera of the android device.

**VideoCapture()** is use to start the camera and captures the video(multiple images)

```python
def sender():
    while True:
        ret, photo = cam.read()
        photo = cv.resize(photo, (640, 480))
        print(photo.shape)
        if ret:
            session.send(np.ndarray.tobytes(photo))
        else: print("No Frame Found")
```
Here is sender function that sends the client the video.

**read()** is use to read the captured images 

**resize()** is use to resize the image

**send(np.ndarray.tobytes(photo))** is use to send the image in the byte forma, here numpy is use to convert the numpy array that contains the value of image in byte.


```python
def receiver():
    framesLost = 0
    print("Connecton Established")
    while True:
        framesLost += 1 
        data = session.recv(100000000)
        if(data == b'finished'):
            print("Finished")
            cam.release()
            session.close()
        else:
            photo =  np.frombuffer(data, dtype=np.uint8)
            if len(photo) == 640*480*3:
                cv.imshow('Hi I am Client', photo.reshape(480, 640, 3))
                if cv.waitKey(1) == 13:
                    session.send(b'finished')
                    cam.release()
                    cv.destroyAllWindows()
                    break
            else:
                print("Lost {} frames".format(framesLost) )
```
Here is receive function that will receive the data from the client.

**frameLost** will count the lost frames during the transmission of the data.

**session.recv(100000000)** here recv function will receive the data in the limit of **100000000**

**cam.release()** is use to close the webcam.

**session.close()** is use to close the session between client and the server.

**np.frombuffer(data, dtype=np.uint8)** here numpy is use to convert the data in byte format to numpy array.

**photo.reshape(480, 640, 3))** is use to reshape the image in the following dimensions.

**cv.waitKey(1) == 13** here **waitKey()** is use to hold the window that contains an image/video and here is **13** is the code for **Enter Key** in the keyboard.

**destroyAllWindows()** this will destroy or close the window as soon as enter key is pressed.

**threading.Thread(target=sender).start()** **threading.Thread(target=receiver).start()** is use to start the thread. Here thread will continue listening for request from the client.

- Thread:

A thread is a sequence of instruction that run independently of the program and of any thread 

- Now we need to create program for Client:

Like the code in server program, we need to code the client program.

```python
soc = socket.socket()
soc.bind(("", 1234))  
server_ip = input(" Enter the Server IP adress -> ")
server_port = input("Enter the Server Port number -> ")

soc.connect((server_ip, int(server_port))) 
soc.send(b"connected")  
camIndex = 0 
cam = cv.VideoCapture(camIndex) 

 server: Client <- Server
def receiver():
    framesLost = 0
    print("Connection Established")
    while True:
        framesLost += 1 
        data = soc.recv(100000000) 
        if(data == b'finished'):
            print("Connection END")
            cam.release() 
            soc.close() 
        else:  
            photo =  np.frombuffer(data, dtype=np.uint8)  
            if len(photo) == 640*480*3: 
                cv.imshow('Hi I am Server', photo.reshape(480,640,3))
                if cv.waitKey(1) == 13: 
                    soc.send(b'finished')
                    cam.release() 
                    cv.destroyAllWindows()  
                    break
            else:
                print("Lost {} frames".format(framesLost) )          
def sender():
    while True:  
        ret, photo = cam.read()
        photo = cv.resize(photo, (640, 480)) 
        print(photo.shape)
        if ret:
            soc.send(np.ndarray.tobytes(photo))
        else: print("No Frame Found")
    cam.release()

threading.Thread(target=receiver).start()
threading.Thread(target=sender).start()
``` 

All code is explained in the server program and client code also works in the same way.

Watch the Demo 👇

{% youtube vb3ZSpvszjw%}

Hope you find this blog insightful.