這是用戶在 2024-8-28 20:23 為 https://readmedium.com/remote-heart-rate-detection-using-webcam-and-50-lines-of-code-2326f6431149 保存的雙語快照頁面,由 沉浸式翻譯 提供雙語支持。了解如何保存?
avatarDmitrii Eliuseev 德米特里·埃利乌塞夫

Summary 摘要

This article discusses a method for remotely detecting heart rate using a webcam and Python code, with a focus on image processing and the use of OpenCV and matplotlib libraries.
本文討論了一種使用網絡攝像頭和 Python 代碼遠程檢測心率的方法,重點是圖像處理和使用 OpenCV 和 matplotlib 庫。

Abstract 摘要

The article begins by describing an Android application that measured heart rate remotely using a smartphone camera, which was met with skepticism and ultimately rejected. The author then decided to test the idea using Python and OpenCV to access the webcam stream. The method involves capturing a skin fragment in the camera's field of view and analyzing the average brightness value of the selected image fragment. The heart rate is then determined by plotting the brightness values against the measurement timestamps using the matplotlib library. The author successfully tested the method and observed a noticeable change in the heart rate after physical exercise. The article concludes by discussing the potential limitations of using a smartphone camera for this purpose and the possibility of analyzing compressed video data.
該文章開始描述了一個使用智能手機相機遠程測量心率的 Android 應用程序,但遭到了懷疑並最終被拒絕。然後,作者決定使用 Python 和 OpenCV 來測試這個想法,以訪問網絡攝像頭流。該方法涉及捕獲攝像頭視野中的皮膚碎片,並分析所選圖像碎片的平均亮度值。然後,通過使用 matplotlib 庫將亮度值與測量時間戳繪製在一起,來確定心率。作者成功地測試了這種方法,並觀察到在體育鍛煉後心率的明顯變化。該文章最後討論了使用智能手機相機進行這一目的的潛在限制以及分析壓縮視頻數據的可能性。

Opinions 意見

  • The author was initially skeptical about the possibility of remotely detecting heart rate using a camera.
    作者最初對使用攝像頭遠程檢測心率的可能性持懷疑態度。
  • The author believes that the method works best with a webcam and a motionless subject.
    作者認為這種方法最適合使用攝像頭和靜止的主題。
  • The author doubts that the results would be reliable if the phone is held in hand.
    作者懷疑如果手持手機,結果是否可靠。
  • The author suggests that motion stabilization algorithms could potentially improve the accuracy of heart rate detection using a smartphone camera.
    作者建議,運動穩定算法有可能提高使用智慧型手機相機進行心率檢測的準確性。
  • The author encourages readers to test the method with compressed video data.
    作者鼓勵讀者使用壓縮的視頻數據來測試這種方法。
  • The author notes that the difference in brightness between heartbeats is less than 0.5%, which is not visible to the naked eye.
    作者指出,心跳之間的亮度差異小於 0.5%,肉眼是看不到的。
  • The author concludes that the method is an interesting and viable way to remotely detect heart rate using a webcam and Python code.
    作者得出結論,該方法是一種有趣且可行的遠程使用網絡攝像頭和 Python 代碼檢測心率的方式。

Remote Heart Rate Detection using Webcam and 50 Lines of Code
使用網絡攝像頭和 50 行代碼進行遠程心率檢測

Once I came across a description of an Android application that measured the heart rate remotely by using the smartphone’s camera. An interesting point was that the Google Play reviewers did not believe in the possibility of such a measurement, and the application was rejected. I don’t know the end of this story, but it became interesting to check whether this is possible. There is no need to make an Android application, it is much easier to test the idea in Python.
有一次我遇到了一個描述一個 Android 應用程式的故事,它可以通過使用智能手機的攝像頭遠程測量心率。有趣的是,Google Play 的評論者們不相信這種測量的可能性,並且拒絕了這個應用程式。我不知道這個故事的結局,但檢查這是否可能變得有趣。沒有必要製作一個 Android 應用程式,用 Python 測試這個想法要容易得多。

Let’s get started. 讓我們開始吧。

Getting the Camera Stream
獲取相機串流

First, we need to get a stream from the webcam, for which I will use OpenCV. The code is cross-platform and can run on both Windows and Linux/OSX.
首先,我們需要從網絡攝像頭獲取一個流,我將使用 OpenCV。這段代碼是跨平台的,可以在 Windows 和 Linux/OSX 上運行。

import cv2
import io
import time
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
cap.set(cv2.CAP_PROP_FPS, 30)
while True:
    ret, frame = cap.read()
    img = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    # Display the frame
    cv2.imshow('Crop', crop_img)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cap.release()
cv2.destroyAllWindows()

The idea of determining the heart rate is that the skin tone changes slightly due to the flow in the blood vessels. So we need a picture crop, which contains only a fragment of the skin:
確定心率的想法是由於血管中的流動,皮膚色調會稍微改變。因此,我們需要一張只包含皮膚片段的圖片裁剪:

x, y, w, h = 800, 500, 100, 100
crop_img = img[y:y + h, x:x + w]
cv2.imshow('Crop', crop_img)

If everything was done correctly, we should get the camera image (blurred for privacy reasons) and crop:
如果一切都做得正確,我們應該能夠獲得攝像頭圖像(出於隱私原因而模糊)並進行裁剪:

Image Processing 圖像處理

Once we have the camera stream, it’s pretty simple. For the selected image fragment, we get the average brightness value and add it to the array along with the measurement timestamp.
一旦我们有了相机流,就很简单了。对于所选的图像片段,我们获取平均亮度值,并将其与测量时间戳一起添加到数组中。

heartbeat_count = 128
heartbeat_values = [0]*heartbeat_count
heartbeat_times = [time.time()]*heartbeat_count
while True:
    ...
    # Update the data and timestamps
    heartbeat_values = heartbeat_values[1:] + [np.average(crop_img)]
    heartbeat_times = heartbeat_times[1:] + [time.time()]

The numpy.average function calculates the average of a two-dimensional array, at the output, we get a number, which is the average brightness of our 100x100 square frame.
numpy.average 函數計算二維數組的平均值,輸出結果是一個數字,該數字表示我們 100x100 正方形框架的平均亮度。

We can display the graph in real-time using the matplotlib library:
我們可以使用matplotlib庫實時顯示圖形:

fig = plt.figure()
ax = fig.add_subplot(111)
while True:
    ...
    ax.plot(heartbeat_times, heartbeat_values)
    fig.canvas.draw()
    plot_img_np = np.fromstring(fig.canvas.tostring_rgb(),
                                dtype=np.uint8, sep='')
    plot_img_np = plot_img_np.reshape(fig.canvas.get_width_height()[::-1] + (3,))
    plt.cla()
    
    cv2.imshow('Graph', plot_img_np)

OpenCV works with images in numpy format, so I have to convert an image using the numpy.fromstring function.
OpenCV 使用 numpy 格式處理圖像,所以我必須使用 numpy.fromstring 函數來轉換圖像。

That’s all. 就這樣。

I have run the program, select such a position so that only a skin fragment is visible in the camera crop area, took The Thinker pose with the head resting on hand — the image should be as motionless as possible. And voila, it really works!
我已經運行了該程序,選擇了這樣一個位置,以便在攝像頭裁剪區域中只有一個皮膚碎片可見,並採取了思考者的姿勢,頭部靠在手上 - 圖像應該盡可能靜止。嗯,真的有效!

It is worth repeating again that the camera is not applied to the skin, we are simply analyzing the overall picture of the person. And it’s amazing that even at this distance, the slight change in skin tone is confidently captured by the camera! As we can see from the graph, the real difference in brightness is less than 0.5% and, of course, it is not visible to the “naked eye”, but confidently distinguishable on the graph. The approximate pulse turned out to be about 75bpm. For comparison, the BPM result from the pulse oximeter:
值得再次重申的是,相機並不應用於皮膚上,我們只是在分析人的整體圖像。令人驚奇的是,即使在這個距離下,相機也能自信地捕捉到皮膚色調的微小變化!從圖表中可以看出,亮度的實際差異小於 0.5%,當然,這對“肉眼”來說是不可見的,但在圖表上可以自信地區分出來。大致脈搏結果約為 75bpm。作為比較,脈搏血氧計的 BPM 結果:

To verify that we get the real heart rate and not any fake signal, like flickering the lightbulb, it is also interesting to check if the heart rate changes depending on the physical exercise. Indeed, I can see it if I put both graphs in the same image — the first graph was taken before the workout, the second after.
為了驗證我們所得到的是真實的心率,而不是任何虛假的信號,例如閃爍的燈泡,檢查心率是否會因身體運動而改變也是很有趣的。事實上,如果我將兩個圖形放在同一張圖像中,我可以看到這一點 - 第一個圖形是在運動前拍攝的,第二個是在運動後拍攝的。

It is easy to see that in the second case the heart rate is higher.
很容易看出,在第二种情况下心率更高。

Conclusion 結論

Oddly enough, but it really works — to be honest, I wasn’t sure about the results. Can it work with a smartphone camera? Well, it depends on many factors. If the phone is placed on the tripod and the person is staying motionless, it can be doable. But if the phone is kept in hand, I strongly doubt that the results will be better than a random number generator ;) On the other side, it may be theoretically possible if the motion stabilization algorithm will be applied to the camera stream before processing, so the question is still open. For the webcam, it is, in general, easier.
奇怪的是,它真的有效-老實說,我對結果不太確定。它能在智能手機相機上運作嗎?嗯,這取決於很多因素。如果手機放在三腳架上,人保持靜止,那麼可能是可行的。但如果手機被拿在手上,我強烈懷疑結果會比隨機數生成器好;) 另一方面,如果在處理之前將運動穩定算法應用於攝像頭流,理論上可能是可行的,所以問題仍然存在。對於網絡攝像頭來說,一般來說更容易。

And since we are analyzing a video stream, a separate question may arise — does this work with compressed video data, is it possible to see the heart rate of a movie or TV actor? I do not know the answer, those who wish can try it on their own. To do this, it is enough to replace the line cap = cv2.VideoCapture(0) in the code with cap = cv2.VideoCapture(“video.mp4”), other code remains the same.
由於我們正在分析視頻流,可能會出現一個單獨的問題 - 是否可以使用壓縮的視頻數據來查看電影或電視演員的心率?我不知道答案,有興趣的人可以自己嘗試。為此,只需將代碼中的cap = cv2.VideoCapture(0)行替換為cap = cv2.VideoCapture(“video.mp4”),其他代碼保持不變。

For those wishing to make more tests, the source code is attached below:

import numpy as np
from matplotlib import pyplot as plt
import cv2
import io
import time
# Camera stream
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 1280)
cap.set(cv2.CAP_PROP_FPS, 30)
# Video stream (optional, not tested)
# cap = cv2.VideoCapture("video.mp4")
# Image crop
x, y, w, h = 800, 500, 100, 100
x, y, w, h = 950, 300, 100, 100
heartbeat_count = 128
heartbeat_values = [0]*heartbeat_count
heartbeat_times = [time.time()]*heartbeat_count
# Matplotlib graph surface
fig = plt.figure()
ax = fig.add_subplot(111)
while True:
    # Capture frame-by-frame
    ret, frame = cap.read()
    img = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    crop_img = img[y:y + h, x:x + w]
    # Update the data
    heartbeat_values = heartbeat_values[1:] + [np.average(crop_img)]
    heartbeat_times = heartbeat_times[1:] + [time.time()]
    # Draw matplotlib graph to numpy array
    ax.plot(heartbeat_times, heartbeat_values)
    fig.canvas.draw()
    plot_img_np = np.fromstring(fig.canvas.tostring_rgb(),
                                dtype=np.uint8, sep='')
    plot_img_np = plot_img_np.reshape(fig.canvas.get_width_height()[::-1] + (3,))
    plt.cla()
    # Display the frames
    cv2.imshow('Crop', crop_img)
    cv2.imshow('Graph', plot_img_np)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
cap.release()
cv2.destroyAllWindows()

Thanks for reading.

Algorithms
Programming
Signal Processing
Python
Recommended from ReadMedium