Abstract 摘要
This study summarizes current trends and future directions in projection mapping technologies. Projection mapping seamlessly merges the virtual and real worlds through projected imagery onto physical surfaces, creating an augmented reality environment. Beyond traditional applications in advertising, art, and entertainment, various fields, including medical surgery, product design, and telecommunications, have embraced projection mapping. This study categorizes recent techniques that address technical challenges in accurately replicating desired appearances on physical surfaces through projected imagery into four groups: geometric registration, radiometric compensation, defocus compensation, and shadow removal. It subsequently introduces unconventional projectors developed to resolve specific technical issues and discusses two approaches for overcoming the inherent limitations of projector hardware, such as the inability to display images floating above physical surfaces. Finally, this study concludes the discussion with possible future directions for projection mapping technologies.
本研究总结了投影映射技术当前趋势和未来发展方向。投影映射通过将投影图像投射到物理表面上,无缝融合虚拟和现实世界,创造一个增强现实环境。除了在广告、艺术和娱乐等传统应用之外,包括医疗手术、产品设计、电信等各个领域都采用了投影映射。本研究将解决通过投影图像在物理表面上准确复制所需外观的技术挑战的最新技术分为四类:几何配准、辐射补偿、失焦补偿和阴影消除。随后,介绍了为解决特定技术问题而开发的非传统投影仪,并讨论了两种克服投影仪硬件固有局限性的方法,例如无法显示漂浮在物理表面上的图像。最后,本研究总结了投影映射技术可能的未来发展方向。
Keywords: projection mapping, augmented reality, projector-camera systems
关键词:投影映射、增强现实、投影相机系统
1. Introduction 1. 简介
Projection mapping (PM) overlays computer-generated imagery onto physical surfaces using projectors, creating an augmented reality (AR) environment where the virtual and real worlds seamlessly merge. These surfaces encompass not only flat and uniform white screens but also general, non-planar, textured surfaces in our surroundings. The PM-based AR provides additional information, such as annotations directly on the target physical surface.1) Furthermore, it can visually alter the material of the physical surface, thus transforming a plaster statue into metallic, transparent, or furry material.2) The PM-based AR is often referred to as spatial AR,3) offering several advantages over other AR display technologies, such as video and optical see-through displays. For instance, PM does not require users to wear or hold displays, such as head-mounted displays or smartphones, thus not restricting the user’s field of view (FOV). Furthermore, it enables multiple users to simultaneously share in-situ AR experiences.
投影映射(PM)通过投影仪将计算机生成的图像叠加到物理表面上,创建一个虚拟与现实世界无缝融合的增强现实(AR)环境。这些表面不仅包括平坦均匀的白色屏幕,还包括我们周围的一般非平面、纹理表面。基于 PM 的 AR 提供额外的信息,如直接在目标物理表面上的注释。此外,它还可以从视觉上改变物理表面的材料,将石膏雕像变成金属、透明或毛茸茸的材料。基于 PM 的 AR 通常被称为空间 AR,提供比其他 AR 显示技术(如视频和光学透视显示)更多的优势。例如,PM 不需要用户佩戴或手持显示设备,如头戴式显示器或智能手机,因此不会限制用户的视野(FOV)。此外,它还使多个用户能够同时共享现场 AR 体验。
Thanks to these advantages, various application fields have been explored beyond typical ones, such as advertising, art, and entertainment.4–6) For example, PM is used to navigate users to target locations by superimposing arrows onto the physical environment.7) Similarly, PM is useful in supporting object searches in physical space by highlighting the searched object.8–12) The highlighting technique is also beneficial in medical surgery, where an invisible emission signal indicating the resection area in a human organ is visualized by projected imagery.13) It can also be used for artwork creation in such a way that projected patterns indicate where to paint on a canvas or where to dig in clay.14–16) Rich graphical information, such as a navigator’s avatar, is projected onto the artwork or its surrounding surfaces for museum guides.17,18) Projected avatars of distant people are also used for tele-communication.19–21) Similarly, human body silhouette projection extends the reaching distance of our body.22–24) The PM on a human face supports makeup.25,26) Apparent material transformation is useful in product design.27–30)
得益于这些优势,各种应用领域已经超越了典型的领域,例如广告、艺术和娱乐。例如,PM 通过在物理环境中叠加箭头来引导用户到目标位置。同样,PM 在支持物理空间中的物体搜索时很有用,通过突出显示搜索到的物体。这种突出显示技术在医疗手术中也很有益,通过投影图像将人体器官中的切除区域的无形发射信号可视化。它还可以用于艺术作品的创作,以这种方式,投影图案指示在画布上绘画或在哪里挖粘土。丰富的图形信息,如导航员的头像,被投影到艺术品或其周围表面上,用于博物馆导游。远程通信中也使用了遥远的人的投影头像。类似地,人体轮廓投影扩展了我们身体的触及距离。人脸上的 PM 支持化妆。明显的材料转换在产品设计中有用。
More conceptually, PM can make any real-world surface, such as a table, wall, and even our skin, visually programmable. When combined with proper sensing technologies, these surfaces become responsive to user actions. Artificial intelligence (AI) technologies further enhance their interactive capability. This enables a paradigm shift in human-computer interaction from inorganic, tangible-based input and output schemas, requiring typical mice, keyboards, touch panels, and 2D monitors, to organic ones. In this paradigm, real-world objects, including our bodies, are covered by so-called smart skins through which we interact with AI in an intimate way. More specifically, we might perceive that AI is symbiotically embedded in our body or exists anywhere surrounding us, which could fundamentally change our perceptual and cognitive model of AI. Such an ultimate application would be achieved when PM becomes ubiquitous, and typical room lights are substituted with projectors.
更概念化地说,PM 可以使任何现实世界的表面,如桌子、墙壁,甚至我们的皮肤,变得视觉可编程。当与适当的传感技术结合时,这些表面对用户动作做出响应。人工智能(AI)技术进一步增强了它们的交互能力。这使得人机交互从无机、基于物理的输入和输出模式,需要典型的鼠标、键盘、触摸屏和二维显示器,转变为有机模式。在这个范式下,现实世界的物体,包括我们的身体,被所谓的智能皮肤所覆盖,通过这些皮肤我们以亲密的方式与 AI 互动。更具体地说,我们可能会感觉到 AI 与我们的身体共生地嵌入,或者存在于我们周围的任何地方,这可能会从根本上改变我们对 AI 的感知和认知模型。当 PM 变得无处不在,典型的室内灯光被投影仪取代时,这种终极应用就会实现。
Technically speaking, these applications work properly only when we can accurately replicate desired appearances on real-world surfaces through projected imagery. However, this is not straightforward, as our projection targets are unconstrained and arbitrary surfaces (e.g., non-planar and textured), often unsuitable for projection. Without careful considerations, the projected image gets severely degraded.
从技术上讲,这些应用程序只有在我们可以通过投影图像准确复制所需外观在现实世界表面时才能正常工作。然而,这并不简单,因为我们的投影目标是未受限制和任意的表面(例如,非平面和有纹理的),通常不适合投影。如果不加仔细考虑,投影图像会严重退化。
First, when a target surface is non-planar, the projected image gets deformed. Thus, geometric registration of a projector to the surface is required to allow a PM system to determine which projector pixel illuminates which surface point. Second, even when pixel correspondence is established, achieving the desired color reproduction on the surface, especially when it is textured, is rarely straightforward. Thus, radiometric compensation is necessary to correct distorted colors. In cases where the surface is non-planar, parts of projected result are defocused and, consequently, appear blurred. Defocus blur should be compensated; otherwise, it significantly reduces high spatial frequency components (or details) of a projected image. Finally, shadows, which occur when a user occludes projected light, significantly reduce the sense of immersion in the AR experience. Therefore, shadow removal is also an important technical issue in PM.
首先,当目标表面非平面时,投影图像会变形。因此,需要对投影仪进行几何配准,以便 PM 系统确定哪个投影仪像素照亮了哪个表面点。其次,即使建立了像素对应关系,在表面上实现所需的颜色再现,尤其是在表面有纹理的情况下,通常并不简单。因此,需要辐射补偿来校正失真的颜色。在表面非平面的情况下,投影结果的部分会失焦,因此看起来模糊。应该补偿失焦模糊;否则,它将显著降低投影图像的高空间频率成分(或细节)。最后,当用户遮挡投影光线时产生的阴影,会显著降低 AR 体验的沉浸感。因此,阴影消除也是 PM 中的一个重要技术问题。
Over the past 25 years, researchers have dedicated their efforts to addressing these technical challenges. They mathematically model the image degradation processes and solve their inverse problems to generate compensation images, allowing them to reproduce desired appearances on target surfaces through projection. However, owing to the technical limitations of projector hardware, such as a limited dynamic range (displayable luminance range) and shallow depth-of-field, compensation images are not always readily displayable. Researchers have successfully overcome these limitations beyond the capabilities of the original projector hardware. Recent trends to achieve this include combining near-eye optics in PM and applying perceptual tricks.
过去 25 年中,研究人员致力于解决这些技术挑战。他们通过数学模型模拟图像退化过程,并解决其逆问题以生成补偿图像,从而通过投影在目标表面上重现所需的视觉效果。然而,由于投影仪硬件的技术限制,如有限的动态范围(可显示亮度范围)和浅景深,补偿图像并不总是容易显示。研究人员已经成功克服了这些超越原始投影仪硬件能力的限制。实现这一目标的最新趋势包括在 PM 中结合近眼光学和运用感知技巧。
This review briefly introduces technical solutions for each challenge and discusses future research directions in PM technologies. Notably, comprehensive surveys on this topic were previously published in 200831) and 2018.32) Therefore, this review focuses on summarizing recent works, specifically those not discussed in the aforementioned literature. In addition, this review concentrates on the technological aspects of PM and does not provide an overview of the trends in recent PM applications.
本综述简要介绍了每个挑战的技术解决方案,并讨论了项目管理技术未来的研究方向。值得注意的是,关于这个主题的全面调查分别在 2008 年和 2018 年发表。因此,本综述侧重于总结近期作品,特别是那些在上文文献中没有讨论的作品。此外,本综述专注于 PM 的技术方面,并不提供近期 PM 应用趋势的概述。
2. Geometric registration
2. 几何配准
The PM system needs to determine which projector pixel incidents on which surface point to display a desired appearance on the surface. Conventional keystone correction addresses this issue only when the surface is flat. However, non-planar surfaces are frequently used in PM, thereby making it necessary to establish proper geometric registration of a projector with the target surface.
投影仪系统需要确定哪个投影仪像素映射到哪个表面点以在表面上显示所需的外观。传统的梯形校正仅当表面平坦时才解决这个问题。然而,在 PM 中,非平面表面经常被使用,因此有必要建立投影仪与目标表面的适当几何注册。
2.1. Projector calibration.
2.1. 投影仪校准。
The geometric relationship between two-dimensional (2D) coordinate value of a projector pixel (x, y) and three-dimensional (3D) coordinate value of corresponding surface point (X, Y, Z) is mathematically described using a pinhole camera model as [x, y, 1]t = PM[X, Y, Z, 1]t, where P and M are 3 × 4 and 4 × 4 matrices, respectively. This model comprises the projector’s intrinsic parameters such as its focal length in P, and the extrinsic parameters that determine the pose of the projector relative to the target surface in M.
二维(2D)投影器像素坐标值(x,y)与对应表面点三维(3D)坐标值(X,Y,Z)之间的几何关系,使用针孔相机模型从数学上描述为[x,y,1] t = PM[X,Y,Z,1] t ,其中 P 和 M 分别是 3×4 和 4×4 矩阵。此模型包括投影器的内在参数,如 P 中的焦距,以及 M 中确定投影器相对于目标表面姿态的外在参数。
Efficient and accurate calibration of these parameters has been successfully established for static PM setups based on a camera calibration framework.33) The standard method involves using a camera to capture projected calibration patterns through which the parameters are then estimated. Recent technologies have also used projector-camera (ProCam) systems for unique setups. For example, Xie et al. introduced a user-friendly calibration technique, allowing a user to use a handheld mobile phone camera in the calibration process.34) Sugimoto et al. proposed an attachment-type system for calibrating intrinsic parameters of a projector in limited space.35) Another group attempted to achieve precise calibration by incorporating a LiDAR (light detection and ranging) sensor with a ProCam system.36) The calibration of multiple projectors using various camera setups, including multiple cameras and a depth camera, is also currently an active area of research.37–39)
高效且精确地校准这些参数已成功建立,适用于基于相机校准框架的静态 PM 设置。 33) 标准方法涉及使用相机捕获投影校准图案,然后通过这些图案估计参数。最近的技术也使用了投影仪-相机(ProCam)系统进行独特设置。例如,谢等人介绍了一种用户友好的校准技术,允许用户在校准过程中使用手持移动电话相机。 34) 菅本等人提出了一种附件式系统,用于在有限空间内校准投影仪的内参。 35) 另一组尝试通过结合 LiDAR(光探测与测距)传感器和 ProCam 系统来实现精确校准。 36) 使用各种相机设置(包括多个相机和深度相机)对多个投影仪进行校准也是当前研究的一个活跃领域。 37–39)
2.2. Dynamic projection mapping.
2.2. 动态投影映射。
The trend in PM research has undergone a dramatic shift from static PM to dynamic PM (DPM) in recent years. In DPM, aligning the projected image with the surface of a moving object is imperative. As a projector’s intrinsic parameters remain constant as long as the lens settings are unchanged, they can be pre-calibrated. Therefore, the primary challenge in achieving DPM is the rapid estimation of extrinsic parameters, specifically, fast tracking of the moving surface. Researchers have tackled this challenge by capturing distinctive visual markers attached to surfaces. The 3D positions of these markers on the surfaces were predetermined. Subsequently, extrinsic parameters could be computed by establishing a relationship between the 3D positions of the markers and their corresponding 2D positions in the captured images.
近年来,项目管理(PM)研究趋势经历了从静态 PM 到动态 PM(DPM)的巨大转变。在 DPM 中,将投影图像与移动物体的表面对齐至关重要。由于投影仪的内在参数在镜头设置不变的情况下保持不变,因此它们可以预先校准。因此,实现 DPM 的主要挑战是快速估计外参数,特别是快速跟踪移动表面。研究人员通过捕捉附着在表面上的独特视觉标记来应对这一挑战。这些标记在表面上的 3D 位置是预先确定的。随后,可以通过建立标记的 3D 位置和它们在捕获图像中相应 2D 位置之间的关系来计算外参数。
Interestingly, multiple research groups have focused on aligning projected images onto deformable surfaces such as cloth by tracking dot array markers captured by an RGB camera.40–43) Typical projectors inevitably introduce noticeable delays in projected images onto a target surface in DPM, even when fast marker tracking is available. Maeda and Koike addressed this problem using deep neural networks (DNNs) for object pose prediction.44) Another issue in marker-based DPM is the visibility of markers. Visible markers under projection significantly reduce a user’s immersion in DPM experience. Notably, this is also a critical concern in low-latency DPM (see Sec. 2.3). Researchers have attempted to reduce the visibility of markers using imperceptible materials (e.g., infrared (IR) ink only detectable using an IR camera)45) or IR LEDs46) as markers (Fig. 1) and have further reduced visibility by projecting complementary colors onto the marker area.47) Edible markers have been developed for projecting images onto foods, and these are also designed to reduce their visibility using transparent materials48) or by embedding markers in internal structures of foods.49)
有趣的是,多个研究小组已经专注于通过跟踪由 RGB 相机捕获的点阵标记,将投影图像对齐到可变形表面,如布料。在 DPM 中,典型的投影仪不可避免地会在将投影图像投射到目标表面时引入明显的延迟,即使有快速标记跟踪。Maeda 和 Koike 使用深度神经网络(DNN)进行物体姿态预测来解决此问题。在基于标记的 DPM 中,另一个问题是标记的可见性。在投影下可见的标记会显著减少用户在 DPM 体验中的沉浸感。值得注意的是,这也是低延迟 DPM 中的一个关键问题(见第 2.3 节)。研究人员已经尝试使用不可察觉的材料(例如,仅使用红外(IR)相机可检测的红外墨水)或 IR LED 作为标记(图 1),并通过在标记区域投射互补颜色来进一步减少可见性。为了在食物上投影图像,已经开发了可食用标记,并且这些标记也设计为使用透明材料或通过将标记嵌入食物的内部结构来减少其可见性。
Marker-less tracking has also been explored. For example, the pose of a rigid target object was robustly tracked based on silhouette information, even in the presence of occlusions caused by the user’s hands, using multiple cameras.50) However, the marker-less approach is generally error-prone. In cases where multiple projectors are used in DPM, tracking errors can result in noticeable misalignments of projected images from different projectors. To address this challenge, Kurth et al. proposed a scalable online solution for their depth camera-based marker-less DPM. Their approach optimizes overlapped projection images by reducing the pixel values from projectors other than the one projecting the finest and brightest pixels, particularly in areas with discontinuities in the depth of the surface point from the projector and in the color of the projected image.51)
无标记跟踪也得到了探索。例如,基于轮廓信息,即使在使用用户手部遮挡的情况下,也能稳健地跟踪刚性目标对象的姿态,使用多个相机。然而,无标记方法通常容易出错。在 DPM 中使用多个投影仪的情况下,跟踪错误可能导致来自不同投影仪的投影图像出现明显的错位。为了应对这一挑战,Kurth 等人提出了一种基于深度相机无标记 DPM 的可扩展在线解决方案。他们的方法通过减少除投影最精细和最亮的像素之外的其他投影仪的像素值,优化了重叠的投影图像,尤其是在投影仪与表面点深度和投影图像颜色不连续的区域。
2.3. Low-latency dynamic projection mapping.
2.3. 低延迟动态投影映射。
Typical projectors with a 60 Hz refresh rate are ill-suited for DPM because the human visual system detects misalignment when the delay from motion to projection exceeds 6–7 ms.52) A promising game-changer overcoming this limitation is a recently developed high-speed projector capable of achieving almost 1,000 frames per second (fps) full-color video projection.53) Alongside the high-speed projector, researchers have used high-speed cameras, nearly 1,000 fps, for tracking 3D pose of a target surface. Marker-based tracking techniques have demonstrated effectiveness in handling rigid surfaces54–56) and non-rigid surfaces.57) In addition, researchers have sought to enhance the quality of projected images while meeting low-latency demands. Nomoto et al. introduced a distributed cooperative approach in multi-projection DPM to ensure that projected images cover the entire surfaces of target objects.58) The same research group elevated the realism of the projected results using a ray tracing technique.59)
典型的 60Hz 刷新率的投影仪不适合 DPM,因为当运动到投影的延迟超过 6-7 毫秒时,人眼视觉系统会检测到错位。 52) 一种有希望的颠覆性技术是最近开发的高速投影仪,能够实现近 1000 帧每秒的全彩视频投影。 53) 与高速投影仪并行,研究人员使用了几乎 1000 帧每秒的高速摄像机来跟踪目标表面的 3D 姿态。基于标记的跟踪技术在处理刚性表面 54–56) 和非刚性表面方面已证明其有效性。 57) 此外,研究人员还寻求在满足低延迟需求的同时提高投影图像的质量。Nomoto 等人引入了一种分布式协作方法,用于多投影 DPM,以确保投影图像覆盖目标对象的整个表面。 58) 同一研究小组使用光线追踪技术提高了投影结果的真实感。 59)
Marker-less tracking is a key element in making DPM more applicable. The most successful field for marker-less DPM is makeup, mainly due to the robust and fast face tracking technologies that have already been established in computer vision research.60) However, addressing fast enough marker-less tracking in other application fields remains a technical challenge, as it necessitates the projection of calibration patterns onto surfaces for estimating the extrinsic parameters. Researchers have proposed projecting calibration patterns and their complementary patterns at high speeds to meet low-latency demands and to make the calibration patterns imperceptible to human observers.61) Another team uses a high-speed IR projector.62) An alternative approach to avoid the pattern projection requirement involves the use of a co-axial high-speed ProCam system where the projector and camera share their optical axes.63)
无标记跟踪是使 DPM 更具应用性的关键要素。无标记 DPM 最成功的领域是化妆,这主要归功于计算机视觉研究中已经建立的强大且快速的人脸跟踪技术。然而,在其它应用领域中实现足够快的无标记跟踪仍然是一个技术挑战,因为它需要将校准图案投射到表面上以估计外参数。研究人员提出了以高速投射校准图案及其互补图案,以满足低延迟需求并使校准图案对人类观察者不可察觉。另一组使用高速红外投影仪。另一种避免图案投射要求的替代方法涉及使用同轴高速 ProCam 系统,其中投影仪和相机共享其光学轴线。
Low-latency DPM can be achieved without the need for high-speed projectors. A promising alternative involves combining a typical projector with a dual-axis galvanometer for rapid redirection of the projector’s illumination direction. Although this approach has the drawback of not being able to quickly adjust the projected image for local pose changes of the target surface, it allows the projected image to smoothly follow the target without noticeable latency, even when the target moves over a large area. For instance, researchers have demonstrated a DPM on a screen mounted on a flying drone.64) A downsized version of the galvanometer-based system can even be worn and used for PM on a moving hand.65) Previous studies have shown that the mentioned drawback can be overcome by substituting the high-speed projector for the typical projector.54,55)
低延迟 DPM 可以在不需要高速投影仪的情况下实现。一种有希望的替代方案是将典型的投影仪与双轴哥尔丹纳结合,以快速改变投影仪的照明方向。尽管这种方法存在无法快速调整投影图像以适应目标表面局部姿态变化的缺点,但它允许投影图像平滑地跟随目标,即使目标在大范围内移动,也不会出现明显的延迟。例如,研究人员已经在安装在飞行无人机上的屏幕上展示了 DPM。 64) 基于哥尔丹纳的缩小版系统甚至可以佩戴并用于移动手部的 PM。 65) 先前的研究表明,通过用典型投影仪代替高速投影仪,可以克服上述缺点。 54,55)
3. Radiometric compensation
3. 放射性补偿
Radiometric or photometric compensation is another essential technique in PM that calculates projector pixel values to display a desired color even on a textured surface, thus making it appear as if it were a uniformly white surface. The typical forward model used in radiometric compensation is described by ci = fi (pj) + ei, where the RGB color vectors of
辐射或光度补偿是 PM 中另一种基本技术,它计算投影仪像素值以在纹理表面上显示所需颜色,从而使它看起来像是一个均匀的白色表面。在辐射补偿中使用的典型正向模型由 c = f(p) + e 描述,其中
To reproduce a desired appearance
为了在 PM 表面重现所需的外观
3.1. Compensation techniques based on hand-crafted color transformation models.
3.1. 基于手工调色模型的对色补偿技术。
Two decades ago, pioneering studies applied linear transformation models as f.31) However, due to the nonlinear nature of color processing in projector hardware, these models suffered from limited compensation accuracy. Thereafter, Grundhöfer demonstrated that a nonlinear model outperforms the linear ones.67,68) Specifically, they used a thin-plate spline (TPS) to approximate nonlinear transformation, which, however, required projecting hundreds of uniformly colored images onto the target surface in advance to calibrate the model parameters.
二十年前,开创性研究将线性变换模型应用于 f。然而,由于投影仪硬件中颜色处理的非线性特性,这些模型在补偿精度方面受到限制。此后,Grundhöfer 证明非线性模型优于线性模型。具体来说,他们使用薄板样条(TPS)来近似非线性变换,但这需要事先将数百张均匀着色的图像投影到目标表面上以校准模型参数。
A recent study has simplified the model complexity using a second-order polynomial.69) It continuously updates the model parameters in a real-time projection-and-capturing feedback loop and adjusts the projected colors accordingly, enabling it to handle changing lighting conditions. Li et al. approximated the nonlinear transformation using a piecewise linear function and significantly reduced the number of projecting calibration patterns by embedding multiple colors into a single pattern, assuming that the spectral reflectance of most real-world materials is smooth.70)
最近的一项研究通过使用二阶多项式简化了模型复杂性。 69) 它在实时投影-捕获反馈循环中持续更新模型参数,并相应地调整投影颜色,使其能够处理变化的照明条件。Li 等人使用分段线性函数近似非线性变换,并通过将多种颜色嵌入到单个图案中,显著减少了投影校准图案的数量,假设大多数真实世界材料的光谱反射率是平滑的。 70)
In addition to efforts solely focused on improving compensation accuracy, other research groups have explored various extension possibilities of radiometric compensation framework. Researchers have concentrated on estimating reflectance properties of target surfaces by decomposing the captured images under different color projections.71,72) The estimated reflectance maps were subsequently used to create novel target appearances, such as reducing color saturation. Amano and their group applied distributed multiple ProCam systems to control the appearance of a surface with view-dependent reflectance properties.73–76) As other extensions, Hashimoto and Yoshimura adapted a radiometric compensation technique for a moving fabric, supporting DPM.77) Pjanic et al. achieved seamless multi-projection displays using TPS-based color transformation model,78) ensuring a seamless transition in the overlapping area of images projected by different projectors.
除了专注于提高补偿准确性的努力之外,其他研究小组还探索了辐射补偿框架的各种扩展可能性。研究人员集中通过在不同颜色投影下分解捕获的图像来估计目标表面的反射率特性。随后,估计的反射率图被用来创建新的目标外观,例如降低色彩饱和度。阿马诺及其团队应用分布式多 ProCam 系统来控制具有视点相关反射率特性的表面外观。作为其他扩展,hashimoto 和 yoshimura 将辐射补偿技术应用于移动织物,支持 DPM。Pjanic 等人使用基于 TPS 的色彩变换模型实现了无缝多投影显示,确保了由不同投影仪投射的图像重叠区域的无缝过渡。
3.2. DNN-based end-to-end compensation techniques.
3.2. 基于深度神经网络的端到端补偿技术。
Very recently, Huang et al. found that DNNs can approximate the nonlinear transformation more accurately than hand-crafted models. They initially demonstrated that DNNs comprising a UNet-like backbone network and an autoencoder subnet, outperformed the classical TPS-based technique79) (Fig. 2). Subsequently, they extended their DNNs to enable geometric registration and radiometric compensation for PM on non-planar surfaces.80) They further improved compensation accuracy by introducing a siamese architecture into their network.81) Other researchers used a differentiable rendering framework in radiometric compensation.82) Handling high-resolution images typically requires long training times and involves high memory costs. Wang et al. mitigated this issue by incorporating a sampling scheme into the network and introducing attention blocks.83) Li et al., in their latest work, reduced the network size by using a network solely for the color transformation of the projector.84) Interestingly, they also demonstrated that a hand-crafted, precise physics-based model of the PM process with limited reliance on neural networks outperformed the end-to-end compensation techniques described above.
最近,黄等人发现,DNNs 可以比手工模型更准确地逼近非线性变换。他们最初证明了由类似 UNet 的骨干网络和自动编码器子网络组成的 DNNs 在经典基于 TPS 的技术 79) (图 2)上表现更优。随后,他们将他们的 DNNs 扩展到能够对非平面表面的 PM 进行几何配准和辐射补偿。 80) 他们通过将 Siamese 架构引入他们的网络,进一步提高了补偿精度。 81) 其他研究人员在辐射补偿中使用了可微渲染框架。 82) 处理高分辨率图像通常需要较长的训练时间和涉及较高的内存成本。王等人通过将采样方案纳入网络并引入注意力块来缓解了这个问题。 83) 李等人在其最新工作中,通过仅使用用于投影仪颜色变换的网络来减小网络大小。 有趣的是,他们还证明了一个手工制作的、基于物理的 PM 过程模型,该模型对神经网络的依赖有限,其性能优于上述所述的端到端补偿技术。
DNNs can be applied to various tasks in addition to radiometric compensation. Huang and Ling demonstrated that their networks could reconstruct the shape of a projected scene and simulate the scene’s appearance under a novel image projection.85) The latter is particularly useful for testing or debugging PM without the requirement for actual PM operations. Erel et al. successfully decomposed scene geometry and view-dependent reflectance properties and estimated the projector’s intrinsic and extrinsic parameters by training neural representations of the scene from multi-view captures under PM with different color patterns.86) They showcased that their DNNs can handle geometric registration and radiometric compensation for novel viewpoints.
深度神经网络可以应用于除了辐射校正以外的各种任务。黄和凌展示了他们的网络能够重建投影场景的形状并模拟场景在新型图像投影下的外观。后者在无需实际 PM 操作的情况下,特别适用于测试或调试 PM。Erel 等人成功分解了场景几何和视点相关的反射特性,并通过在具有不同颜色模式的 PM 下从多视角捕获中训练场景的神经表示来估计投影仪的内在和外在参数。他们展示了他们的 DNN 可以处理新型视点的几何配准和辐射校正。
4. Defocus compensation 4. 虚化补偿
As projectors are designed to emit maximum brightness through their lenses, they have a large aperture size, resulting in a shallow depth-of-field (DOF). The typical forward model of defocus blur is described by I′ = K * I, where I, I′, and K represent a projected image without suffering from defocus, a defocused result, and a spatially varying 2D defocus kernel, respectively. In this equation, * represents a 2D convolution process. Deblurring the projected result is achieved by computing the inverse of the forward model. However, standard algorithms such as Wiener filter are unsuitable because the dynamic range of a projector device is not infinite (e.g., the maximum luminance is limited, and negative light is physically not displayable).
投影仪设计为通过镜头发射最大亮度,因此具有较大的光圈尺寸,导致景深(DOF)较浅。典型的散焦模糊前向模型由 I′ = K * I 描述,其中 I、I′ 和 K 分别代表未受散焦影响的投影图像、散焦结果和空间变化的二维散焦核。在这个方程中,* 代表二维卷积过程。通过计算前向模型的逆来去模糊投影结果。然而,如维纳滤波器等标准算法不适用,因为投影仪设备的动态范围不是无限的(例如,最大亮度有限,负光在物理上无法显示)。
A classical study solved this problem using an iterative, constrained steepest-descent algorithm.87) A recent work introduced a non-iterative technique that simply enhances the pixel intensities around the edge areas that are lost due to defocus blur, resulting in reduced computational time.88) These techniques require a dot pattern projection to obtain spatially varying blur kernels every time either the projector or the surface moves. Kageyama et al. recently addressed this issue using DNNs89,90) (Fig. 3). Specifically, their DNNs estimated the blur kernels from the PM result of a natural image and generated the projection image compensating for defocus blur.
经典研究使用迭代、约束最速下降算法解决了这个问题。 87) 最近的一项工作介绍了一种非迭代技术,该技术简单地增强了由于失焦模糊而丢失的边缘区域的像素强度,从而减少了计算时间。 88) 这些技术需要点图案投影来获取每次投影仪或表面移动时的空间变化模糊核。Kageyama 等人最近使用 DNNs 解决了这个问题 89,90) (图 3)。具体来说,他们的 DNNs 从自然图像的 PM 结果中估计模糊核,并生成补偿失焦模糊的投影图像。
The compensation capacity of the software-based solutions mentioned above is restricted by limited dynamic range of projector hardware. Researchers have developed hardware-based solutions to overcome this limitation. Xu et al. proposed a multifocal projector comprising an electrically focus-tunable lens (ETL) and a synchronized high-speed projector.91) They modulate the focal length of the ETL at more than 60 Hz, thus making it imperceptible to human observers, and project images precisely when the focusing distance of the projector corresponds to the target surface. The same setup also achieved a varifocal projector in which ETL’s focal length was constantly adjusted to match the target surface.92) Although a large aperture ETL would be suitable for these systems, the response time of such ETLs is limited. The ETL is made of an optical fluid sealed off by an elastic polymer membrane. An actuator ring exerts pressure on the outer zone of the container, changing the curvature of the lens. The response time limitation is caused by the rippling of the optical fluid after actuation. Researchers demonstrated that input signals computed using sparse optimization can speed up the response time.93)
上述基于软件的解决方案的补偿能力受到投影机硬件动态范围有限的限制。研究人员已经开发了基于硬件的解决方案来克服这一限制。徐等人提出了一种多焦点投影仪,包括一个电聚焦可调镜头(ETL)和一个同步的高速投影仪。他们在超过 60 Hz 的频率下调制 ETL 的焦距,使其对人类观察者来说难以察觉,并在投影仪的聚焦距离与目标表面相对应时精确投影图像。同样的设置还实现了一种变焦投影仪,其中 ETL 的焦距不断调整以匹配目标表面。尽管大孔径 ETL 适合这些系统,但此类 ETL 的响应时间有限。ETL 由一个弹性聚合物膜密封的光学流体制成。一个执行器环对容器的外部区域施加压力,改变镜头的曲率。响应时间限制是由执行后光学流体的波动引起的。 研究人员证明,使用稀疏优化计算输入信号可以加快响应时间。 93)
Other hardware-based solutions control the waveform of the projected light. Li et al. proposed optimizing the diffractive optical element to preserve the high spatial frequency components of the projected image over various distances, thereby extending the DOF of the projector.94) Other researchers have proposed spatially adaptive focal projection, coining the term “focal surface projection” to describe their approach, using a phase-only spatial light modulator. This approach enables focusing on all parts of a non-planar target surface.95)
其他基于硬件的解决方案控制投影光波的波形。李等人提出了优化衍射光学元件,以保留投影图像在不同距离上的高空间频率成分,从而扩展投影机的景深。 94) 其他研究人员提出了空间自适应焦点投影,创造了“焦点表面投影”这一术语来描述他们的方法,使用相位仅空间光调制器。这种方法能够聚焦于非平面目标表面的所有部分。 95)
5. Shadow removal 5. 去阴影
Cast shadows significantly degrade the sense of immersion in PM. Previous studies removed shadows using synthetic aperture approaches. Specifically, they spatially distributed multiple projectors to ensure that users do not simultaneously occlude a projection target from all projectors. Once either an occluder or its shadow is detected by cameras, the system compensates for the shadow by illuminating that area from an unoccluded projector.96) Although they computed the projection images for all projectors on a single central server, the recent research trend has shifted toward applying cooperative distributed algorithms since around 2015.97) Uesaka and Amano proposed a technique in which multiple co-axial ProCam systems cooperatively remove shadows.98) Nomoto et al. demonstrated shadow removal in DPM with multiple high-speed projectors using a cooperative algorithm.58) However, these synthetic aperture approaches suffer from a delay in computational compensation process. In other words, a shadow cannot be perfectly removed while an occluder is moving.
投影产生的阴影显著降低了 PM 的沉浸感。先前的研究通过合成孔径方法去除阴影。具体来说,他们通过空间分布多个投影仪来确保用户不会被所有投影仪同时遮挡投影目标。一旦摄像头检测到遮挡物或其阴影,系统就会通过从未被遮挡的投影仪照亮该区域来补偿阴影。尽管他们计算了所有投影仪在单个中央服务器上的投影图像,但自 2015 年以来,最新的研究趋势已经转向应用协作分布式算法。Uesaka 和 Amano 提出了一种技术,其中多个同轴 ProCam 系统协作去除阴影。Nomoto 等人使用协作算法在 DPM 中展示了使用多个高速投影仪去除阴影。然而,这些合成孔径方法在计算补偿过程中存在延迟。换句话说,当遮挡物移动时,阴影无法被完美去除。
By contrast, optical approaches achieve delay-free shadow removal and have also attracted significant attention from researchers. Hiratani et al. applied a large-format retrotransmissive plate to project images onto a surface from wide viewing angles99,100) (Fig. 4). The retrotransmissive plate collects the light rays emitted from a point in space at a plane-symmetrical position with respect to it. They prepared a white diffuse object (proxy object) with a shape that is plane-symmetrical to the projection target and placed the target and proxy objects in a plane-symmetrical arrangement with respect to retrotransmissive plate. When an image is projected onto the proxy object, the reflected light rays pass through the retrotransmissive plate and converge on the target object. Consequently, the appearance of the proxy object is duplicated on the target object’s surface. When the size of the retrotransmissive plate is sufficiently large relative to an occluder, shadowless PM is achieved without the need for the shadow removal computations used in conventional synthetic aperture approaches.
相比之下,光学方法实现了无延迟的阴影消除,并也引起了研究者的广泛关注。Hiratani 等人应用了一种大尺寸反透射板,从宽视角将图像投射到表面上(图 4)。反透射板收集来自空间中某点的光线,相对于它处于平面对称位置。他们准备了一个白色扩散物体(代理物体),其形状与投影目标平面对称,并将目标和代理物体放置在相对于反透射板的平面对称排列中。当图像投射到代理物体上时,反射的光线穿过反透射板并聚焦在目标物体上。因此,代理物体的外观在目标物体表面上被复制。当反透射板相对于遮挡物足够大时,无需使用传统合成孔径方法中使用的阴影消除计算,即可实现无阴影的 PM。
The above optical solution is restricted to static DPM because the proxy and target objects must be placed at the plane-symmetrical positions. Other researchers have overcome this limitation by moving the proxy object using a robotic arm to match its pose with the target object.101) The same research group also proposed using a volumetric display102) and light field display103) instead of placing a physical proxy object to generate light rays as if they were emitted from the surface of a proxy object whose pose matches that of the target object.
上述光学解决方案仅限于静态 DPM,因为代理对象和目标对象必须放置在平面对称的位置。其他研究人员通过使用机械臂移动代理对象来克服这一限制,使其姿态与目标对象匹配。该研究小组还提出了使用体积显示①和光场显示②,而不是放置物理代理对象来生成光线,就像它们是从与目标对象姿态匹配的代理对象表面发出的。
6. Unconventional projectors
6. 非传统投影仪
As discussed in the previous sections, unconventional projectors such as those with ETLs and high-speed ones can fundamentally resolve the specific technical issues that could not be addressed using standard projectors. This section introduces three types of unconventional projectors each of which has been currently explored by multiple research groups.
如前几节所述,非传统投影仪,如带有 ETLs 和高速的投影仪,可以从根本上解决标准投影仪无法解决的具体技术问题。本节介绍了三种非传统投影仪,每种都已被多个研究小组探索过。
6.1. Wearable projectors.
6.1. 便携式投影仪。
Following pioneering work,104) several researchers have explored PM using wearable projectors.105,106) The recent trend of downsizing projector hardware, coupled with bright light sources such as LEDs and lasers, has driven the research in this direction. Wearable projectors are valuable for PM onto nearby surfaces or the user’s body. For example, a tiny projector was used as a display component of a smartwatch, enabling a user to interact with the overlaid smartwatch image contents on their arm.107) Another study combined a wearable projector with a pan-tilt mirror and high-speed camera to make projected images follow a moving target surface without perceivable latency.65) Head-mounted setups were also explored, wherein the distance between a projector and user’s eye is reduced, enabling nearly occlusion-free PM.108)
在开创性工作的基础上,多位研究人员已经探索了使用可穿戴投影仪进行投影映射(PM)。近期投影仪硬件小型化的趋势,以及 LED 和激光等明亮光源的配合,推动了这一方向的研究。可穿戴投影仪对于在附近表面或用户身体上进行投影映射非常有价值。例如,一个小型投影仪被用作智能手表的显示组件,使用户能够与其手臂上叠加的智能手表图像内容进行交互。另一项研究将可穿戴投影仪与全景倾斜镜和高速摄像机结合,使投影图像能够无感知延迟地跟随移动的目标表面。还探索了头戴式设置,其中投影仪与用户眼睛之间的距离减小,实现了几乎无遮挡的投影映射。
A current research trend involves combining actuated head-mounted projectors with head-mounted displays. Wang et al. proposed attaching a projector to a virtual reality (VR) headset.109) Their system projected VR scenes that the headset user is watching onto the floor around them, enabling them to share their VR experiences with others. Hartmann et al. suggested using a head-mounted projector with an optical see-through AR headset and demonstrated the sharing of augmented image contents displayed in the headset with people in the vicinity through projected imagery.110) They also proposed displaying auxiliary information and user interface widgets with the head-mounted projector to support interaction with the image contents displayed in the headset.
当前研究趋势涉及将驱动式头戴投影仪与头戴式显示器相结合。王等人提出将投影仪附着在虚拟现实(VR)头盔上。 109) 他们的系统将头盔用户正在观看的 VR 场景投射到他们周围的地面,使他们能够与他人分享他们的 VR 体验。哈特曼等人建议使用带有光学透视增强现实(AR)头盔的头戴式投影仪,并通过投影图像展示了头盔中显示的增强图像内容与附近的人的共享。 110) 他们还提出使用头戴式投影仪显示辅助信息和用户界面小部件,以支持与头盔中显示的图像内容的交互。
6.2. Omnidirectional projectors.
6.2. 全向投影仪。
The FOV of a typical projector is limited, necessitating the use of multiple projectors to achieve large-area PM. An omnidirectional projector, using a fisheye lens with almost a 180-degree FOV, presents a potential solution for this issue. A research group proposed an omnidirectional projector and demonstrated various PM applications using it.111,112) The geometric registration of an omnidirectional projector is non-trivial because the pinhole camera model is no longer valid. The researchers addressed this problem using a co-axial approach in which a projector and camera share their optical axes using a beam-splitter before the fisheye lens. Their co-axial omnidirectional ProCam system can project images onto physical surfaces without distortion, on which visual markers are attached.
典型投影仪的视场角有限,需要使用多个投影仪来实现大面积的 PM。一个全向投影仪,使用几乎 180 度视场角的鱼眼镜头,为解决这个问题提供了一个潜在方案。一个研究小组提出了一种全向投影仪,并展示了使用它进行各种 PM 应用。全向投影仪的几何配准非同寻常,因为针孔相机模型不再有效。研究人员使用了一种共轴方法来解决这一问题,其中投影仪和相机在鱼眼镜头之前使用分束器共享它们的视轴。他们的共轴全向 ProCam 系统可以将图像投射到物理表面上,不产生扭曲,并在其上附加视觉标记。
Yamamoto et al. implemented an omnidirectional ProCam system using another unique approach.113) They proposed a monocular ProCam system in which the projector and camera share the same objective lens. Using relay optics, they optically transferred the image panels of the camera and projector to the focal point of the objective lens, resulting in overlaid image panels. The overlaid pixels have sensing and displaying capabilities. They realized an omnidirectional ProCam system using a fisheye lens as the objective within this framework. Furthermore, they showcased the high scalability of their approach by implementing a high dynamic range ProCam system using a traditional double modulation framework.114,115)
山本等人采用一种独特的方法实现了全向 ProCam 系统。他们提出了一种单目 ProCam 系统,其中投影仪和相机共享同一个物镜。通过使用中继光学,他们将相机和投影仪的图像面板光学地转移到物镜的焦点,从而实现了叠加的图像面板。叠加的像素具有感知和显示功能。他们在这个框架内使用鱼眼镜头作为物镜实现了全向 ProCam 系统。此外,他们通过实现一个使用传统双调制框架的高动态范围 ProCam 系统,展示了他们方法的高可扩展性。
6.3. Visible light communication projectors.
6.3. 可见光通信投影仪。
Embedding invisible code independently in each projected pixel enables the control of electronic devices with photo sensors within the projector’s FOV while simultaneously presenting meaningful images to human observers who remain unaware of the embedded information. In other words, the projector has the capability for visible light communication at the pixel level. This can be achieved by modulating the projected light intensity at a very high speed such as 1 MHz, which is much higher than critical flicker fusion frequency of the human visual system. While pioneering work was published in 2007,116) where fixed information was embedded in grayscale images, this topic is still actively explored by multiple research groups.
将不可见代码独立嵌入到每个投影像素中,可以在投影仪的视场内控制带有光电传感器的电子设备,同时向人类观察者展示有意义的图像,而观察者对嵌入的信息浑然不觉。换句话说,投影仪具有在像素级别进行可见光通信的能力。这可以通过以非常高的速度(如 1 MHz)调制投影光强度来实现,这比人类视觉系统的临界闪烁融合频率要高得多。虽然开创性的工作在 2007 年就已经发表, 116) 其中固定信息被嵌入到灰度图像中,但这个主题仍然被多个研究小组积极研究。
A recent study achieves embedding information in full-color images that can be updated interactively.117) It was demonstrated that the embedded information controls multiple robots,118,119) and wearable haptic displays120) (Fig. 5), in cooperation with graphical images. Although these systems read embedded information using photo sensors, Kumar et al. demonstrated that a high-speed camera can simultaneously read the information embedded in different pixels.121) Researchers have developed a projector emitting RGB as well as IR light, embedding information in the IR channel.122)
最近的一项研究实现了在彩色图像中嵌入可交互更新的信息。 117) 已证明嵌入的信息可以控制多个机器人, 118,119) 以及可穿戴触觉显示器 120) (图 5),并与图形图像合作。尽管这些系统使用光电传感器读取嵌入的信息,但 Kumar 等人证明了高速相机可以同时读取不同像素中嵌入的信息。 121) 研究人员开发了一种发射 RGB 以及红外光的投影仪,在红外通道中嵌入信息。 122)
7. Overcoming technical limitations
7. 克服技术限制
Projector hardware has inherent limitations that cannot be addressed by projector devices alone. This section summarizes two approaches to tackle these limitations; one combines near-eye optics in PM, and the other uses perceptual tricks.
投影仪硬件存在无法仅通过投影仪设备解决的固有局限性。本节总结了两种应对这些局限性的方法;一种结合了 PM 中的近眼光学,另一种使用感知技巧。
7.1. Combining near-eye optics.
7.1. 近眼光学组合。
In typical PM, projectors alter the appearance of target surfaces, although displaying images floating above physical surfaces is not feasible. Stereoscopic PM technology overcomes this limitation, allowing users to perceive 3D objects that appear to float above physical surfaces with arbitrary shapes. These effects are achieved through the tracking of an observer’s viewpoint, rendering perspectively correct images with appropriate disparity for each eye, and projecting these two images in a time-sequential manner within each frame. The projected images are viewed through active-shutter glasses equipped with liquid crystal shutters, which prevent image interference between the two eyes. Researchers have recognized the potential of stereoscopic PM in various fields, including museum guides,18) product design,30) architecture planning,123) and teleconferencing.124)
在典型的项目管理中,投影仪改变目标表面的外观,尽管在物理表面上方显示图像是不可行的。立体项目管理技术克服了这一限制,使用户能够感知似乎漂浮在任意形状物理表面上的 3D 对象。这些效果是通过跟踪观察者的视角,渲染具有适当视差且视角正确的图像,并在每个帧中以时间顺序投影这两幅图像来实现的。投影的图像通过配备液晶快门的主动式快门眼镜观看,这可以防止两只眼睛之间的图像干扰。研究人员已经认识到立体项目管理在博物馆导游、产品设计、建筑规划、以及远程会议等各个领域的潜力。
A recent work applied the principle of stereoscopic PM to alter the appearance of a mirror surface.125) This technique does not directly project images onto a mirror surface; instead, it projects images onto diffuse surfaces that are visible to an observer through the mirror. With stereoscopic PM, the distance of the projected diffuse surfaces matches that of the mirror surface.
最近的一项工作应用了立体 PM 原理来改变镜面外观。 125) 这种技术并非直接将图像投影到镜面上;相反,它将图像投影到观察者通过镜子可见的漫反射表面上。使用立体 PM,投影的漫反射表面距离与镜面表面相匹配。
Typical stereoscopic PM technology only addresses binocular cues and cannot provide accurate focus cues, leading to a vergence-accommodation conflict (VAC) that causes significant discomfort, fatigue, and distorted 3D perception for the observer. Recent studies have tried to mitigate VAC. Fender et al. optimized the placement of the displayed 3D objects such that the depth difference becomes small between the projected physical surface and displayed objects.126) Kimura et al. proposed a multifocal stereoscopic PM to address VAC.127) They attached ETLs to active-shutter glasses and applied fast and periodical focal sweeps to ETLs, causing the “virtual image” (as an optical term) of every part of the real scene seen through ETLs to move back and forth during each sweep period. In each frame, the 3D objects were projected from a synchronized high-speed projector at the exact moment that the virtual image of the projected imagery on a real surface is located at a desired distance from ETLs.
典型立体 PM 技术仅处理双眼线索,无法提供准确的聚焦线索,导致视觉辐辏调节冲突(VAC),这会引起观察者显著的不适、疲劳和扭曲的 3D 感知。最近的研究试图减轻 VAC。Fender 等人优化了显示 3D 物体的位置,使得投影的物理表面和显示物体之间的深度差异变得很小。 126) Kimura 等人提出了一种多焦点立体 PM 来解决 VAC。 127) 他们将 ETLs 附着在主动快门眼镜上,并对 ETLs 进行快速和周期性的聚焦扫描,导致通过 ETLs 看到的真实场景的每个部分的“虚拟图像”(作为一个光学术语)在每个扫描周期内来回移动。在每一帧中,3D 物体由同步的高速投影仪在虚拟图像位于 ETLs 所需距离处时投影出来。
Using ETLs as eyeglasses in conjunction with a synchronized high-speed projector creates other novel vision experiences that cannot be achieved using projectors alone. Ueda et al. proposed using a high-speed projector to illuminate a real scene rather than overlaying images onto it. This approach allows for spatially non-uniformly defocused real-world appearances, irrespective of the distance from the user’s eyes to observed real objects128) (Fig. 6). They achieved this by periodically modulating the focal lengths of the glasses at a rate exceeding 60 Hz. During a specific phase when optical power of ETLs is too high for a user to adjust their vision to focus on the scene, one part of the scene intended to appear blurred is illuminated by the projector, whereas another part intended to appear focused is illuminated during a different phase. This process realizes the spatial defocusing effect that can be used for gaze navigation.129) Based on a similar principle, Ueda et al. used two ETLs for each eye for spatial zooming, where a part of a scene is zoomed in.130)
使用 ETLs 作为眼镜与同步的高速投影仪结合,创造出仅使用投影仪无法实现的全新视觉体验。上田等人提出使用高速投影仪照亮真实场景,而不是将其图像叠加到场景上。这种方法可以实现空间非均匀的模糊真实世界外观,无论用户眼睛与观察到的真实物体之间的距离如何(图 6)。他们通过以超过 60Hz 的速率周期性地调制眼镜的焦距来实现这一点。在 ETLs 的光学功率过高,用户无法调整视力以聚焦于场景的特定阶段,投影仪照亮场景中预定模糊的部分,而在不同的阶段照亮预定聚焦的部分。这个过程实现了可用于视线导航的空间模糊效果。基于类似原理,上田等人使用每只眼睛两个 ETLs 进行空间缩放,其中场景的一部分被放大。
Although PM can be used to visually alter the material of a real surface, it cannot simultaneously reproduce view-dependent effects such as specular reflections for multiple observers. Hamasaki et al. addressed this issue by incorporating optical see-through displays into PM.131) They displayed the view-dependent components on an optical see-through display worn by each observer. They also demonstrated that the system can extend the dynamic range, i.e., contrast, of the displayed results. Another intriguing research direction at the intersection of PM and optical see-through displays has emerged. Itoh et al. realized a lightweight optical see-through display comprising a screen and thin optics onto which a pan-tilt telescopic projector installed in the environment provides images on their see-through display.132) They have continued to improve the system, for example, by developing thinner optics using holographic optical elements (HOE)133) and achieving a low-latency PM on the screen using a 2D lateral effect position sensor.134)
尽管 PM 可以用来视觉上改变真实表面的材料,但它不能同时为多个观察者再现视点相关的效果,如镜面反射。Hamasaki 等人通过将光学透视显示器集成到 PM 中解决了这个问题。他们在每个观察者佩戴的光学透视显示器上显示了视点相关的组件。他们还证明了该系统可以扩展显示结果的动态范围,即对比度。在 PM 和光学透视显示器交叉的研究方向中,又出现了一个有趣的研究方向。Itoh 等人实现了一个由屏幕和薄光学元件组成的轻量级光学透视显示器,环境中的全景倾斜望远镜投影仪在他们的透视显示器上提供图像。他们继续改进系统,例如,通过开发使用全息光学元件(HOE)的更薄光学元件,并在屏幕上使用 2D 横向效应位置传感器实现低延迟的 PM。
7.2. Perceptual tricks. 7.2. 感知技巧。
Considering human perceptual properties is useful in PM. Even if a desired appearance is not physically reproducible by projected imagery onto a physical surface, PM is considered successful when observers perceive that the desired appearance is displayed. Researchers have explored the possibility of reproducing physically unfeasible appearances in PM in a perceptually equivalent manner. As the ultimate example, Sato et al. made the motion trajectory of a real object appear bent using a high-speed projector, although it physically moved in a rectilinear manner.135)
考虑到人类感知特性在项目管理中很有用。即使期望的外观无法通过投影图像到物理表面来物理上再现,只要观察者感知到期望的外观被显示出来,项目管理就被认为是成功的。研究人员已经探索了在项目管理中以感知等效的方式再现物理上不可行的外观的可能性。作为最终例子,Sato 等人使用高速投影仪使真实物体的运动轨迹看起来弯曲,尽管它在物理上以直线方式移动。 135)
The performance in radiometric compensation (see Sec. 3) is significantly restricted by the limited color space and dynamic range of projectors. Researchers have addressed this issue by leveraging the nonlinear properties of the human visual system. Akiyama et al. developed a unique radiometric compensation technique using the color constancy of the human visual system and demonstrated that their method can perceptually enlarge the displayable color space beyond the capability of the projector devicealone.136) Similarly, Nagata and Amano used glare illusion to perceptually enhance glossiness of a projected result.137)
辐射补偿(见第 3 节)的性能受到投影仪有限色域和动态范围的显著限制。研究人员通过利用人眼视觉系统的非线性特性来解决这个问题。秋山等人开发了一种独特的辐射补偿技术,利用人眼视觉系统的颜色恒常性,并证明了他们的方法可以在感知上扩大可显示色域,超越投影仪设备的单独能力。同样,长谷川和天野利用眩光错觉来感知增强投影结果的光泽度。
Although deforming an actual surface is not physically possible, Kawabe et al. demonstrated that overlaying monochrome movement patterns onto a 2D static textured object, such as a painted picture, induces illusory movement perception.138) Specifically, human observers perceive the projected results as if the static picture is moving along with the projected movements. Recent studies have extended this technique to DPM, achieving the modulation of the perceived stiffness of fabric139) and the low-latency deformation of handheld rigid objects using a high-speed projector.140) Fukiage et al. developed a computational framework to optimize the projected movement pattern to maximize the illusory effect.141) Okutani et al. found that 3D deformation is also possible when combining stereoscopic PM.142)
尽管在实际表面上变形在物理上是不可能的,Kawabe 等人证明了将单色运动模式叠加到二维静态纹理对象上,例如一幅画,会诱导出错觉运动感知。具体来说,人类观察者会将投影结果感知为静态图片随着投影的运动而移动。最近的研究将这项技术扩展到 DPM,实现了通过高速投影仪调节感知的布料刚度以及手持刚性物体的低延迟变形。Fukiage 等人开发了一个计算框架来优化投影运动模式以最大化错觉效果。Okutani 等人发现,结合立体 PM 时,三维变形也是可能的。
Researchers have experimentally proven that the depth perception of a physical surface can be manipulated without applying stereoscopic PM. Kawabe et al. found that adding shadows induces the perceived depth modulation of a 2D picture.143) Schmidt et al. investigated how projected color temperature, luminance contrast, and blur affect the perceived depth of the projected surface through a series of psychophysical experiments.144) They found that perceived depth can be influenced by projected illusions, and in particular, an increase in the luminance contrast between an object and its surroundings made the object appear close to the observer.
研究人员通过实验证明,在不应用立体 PM 的情况下,可以操纵物理表面的深度感知。Kawabe 等人发现,添加阴影可以诱导二维图像的感知深度调制。 143) Schmidt 等人通过一系列心理物理实验研究了投影颜色温度、亮度对比度和模糊度如何影响投影表面的感知深度。 144) 他们发现,感知深度可以受到投影错觉的影响,特别是物体与其周围环境亮度对比度的增加使物体看起来更靠近观察者。
The PM is a form of visual media, primarily providing visual perception without directly engaging other sensory modalities. Researchers have sought to overcome this limitation by exploring cross-modal interactions between visual stimuli and other sensory experiences. In a recent study, it was demonstrated that altering a user’s finger position in response to the deformation of a physically touched surface in stereoscopic PM can induce a change in the perceived shape of the touched surface.145) Another study reported that haptic sensations can be induced by providing various visual effects to a virtual hand projected onto physical surfaces. The projected hand movement is determined by magnifying a user’s physical hand movement on a touch panel. When the projected hand movement is inconsistent with the actual hand movement, users reported experiencing haptic sensations.146) Researchers have also recently investigated how the appearance manipulation of food in PM affects its taste. Suzuki et al. projected dynamic boiling texture onto foods and found that it influences the perceived taste, such as saltiness.147) Fujimoto demonstrated that modifying the color saturation or the intensity at the highlight region of food enhances the perceived deliciousness.148)
总理是一种视觉媒体形式,主要提供视觉感知,而不直接涉及其他感官模式。研究人员通过探索视觉刺激与其他感官体验之间的跨模态交互来克服这一局限性。在一项最近的研究中,研究人员证明了通过在立体视觉媒体中触摸变形的表面来改变用户的指尖位置,可以引起触摸表面感知形状的变化。另一项研究报道称,通过向投射到物理表面的虚拟手提供各种视觉效果,可以产生触觉感觉。投影的手部运动是通过放大用户在触摸面板上的物理手部运动来确定的。当投影的手部运动与实际手部运动不一致时,用户报告说他们经历了触觉感觉。研究人员最近还研究了在视觉媒体中食物外观操纵如何影响其味道。铃木等人将动态沸腾纹理投影到食物上,发现它影响了感知的味道,如咸味。 福島本展示了通过调整食物高光区域的色彩饱和度或强度可以增强其美味感知。 148)
8. Future directions 8. 未来方向
A future direction, derived by a simple extrapolation of current research trends, is the development of a low-latency DPM technique that simultaneously addresses radiometric compensation, deblurring, and shadow removal issues. This goal is not particularly challenging when projection targets are rigid body objects tracked using markers, as a recent study has already addressed a part of the required issues.58) Alternatively, achieving the goal for non-rigid surfaces without markers is not simple. Furthermore, a recent study revealed that even with a 1,000-fps projector and high-speed camera, achieving a motion-to-projection latency of less than 6 ms, the misalignment between a moving target object and projected image is sometimes noticeable.149) Therefore, achieving a much lower latency than 6 ms is crucial in DPM. Anticipating this, Nakagawa and Watanabe developed a 5,600 fps projector.150)
未来一个方向,通过简单外推当前研究趋势得出,是开发一种低延迟 DPM 技术,该技术同时解决辐射补偿、去模糊和阴影消除问题。当投影目标是使用标记跟踪的刚体物体时,这个目标并不特别具有挑战性,因为最近的研究已经解决了所需问题的一部分。 58) 另一方面,在没有标记的非刚性表面上实现这一目标并不简单。此外,最近的一项研究显示,即使使用 1,000 fps 投影仪和高速摄像机,要实现小于 6 毫秒的运动到投影的延迟,移动目标物体与投影图像之间的错位有时是明显的。 149) 因此,在 DPM 中实现比 6 毫秒更低的延迟至关重要。为此,中川和渡边开发了一种 5,600 fps 投影仪。 150)
Technical issues persist even in static PM scenarios. Recently, an intriguing grand challenge in display technologies was coined: achieving “perceptual realism”, producing imagery indistinguishable from real-world 3D scenes.151) The most significant discrepancy between the PM and real-world 3D scenes arises from the fact that PM works properly only in a dark environment. The PM in a dark room tends to induce a self-luminous impression for a projected object.152) Researchers have recently started addressing this issue by substituting projectors for room lights and reproducing the environmental illumination while excluding the projection target with the projectors.153–156) As there are still many technical difficulties including the above-mentioned dark room constraint in realizing perceptual realism in PM, this research topic will be intensively explored in the next decade.
技术问题即使在静态 PM 场景中也持续存在。最近,在显示技术领域提出了一个引人入胜的宏大挑战:实现“感知真实感”,产生与真实世界 3D 场景无法区分的图像。 151) PM 与真实世界 3D 场景之间最显著的差异在于,PM 仅在黑暗环境中才能正常工作。在黑暗房间中的 PM 往往会给投影物体带来自发光的印象。 152) 研究人员最近开始通过用房间灯光代替投影仪并重现环境光照,同时用投影仪排除投影目标来解决这一问题。 153–156) 由于在实现 PM 中的感知真实感方面仍存在许多技术难题,包括上述黑暗房间限制,因此这一研究主题将在未来十年内得到深入研究。
Integrating projection systems of other sensory modalities (referred to as X) into full-color PM (i.e., RGB-X PM) to expand user experiences is another promising research direction. For instance, researchers have combined a typical RGB projector with a thermal projector capable of changing the direction of a far IR light spot, simultaneously providing visible and thermal sensations on a user’s body.157) In addition, an aerial vibrotactile display based on an ultrasound phased array can provide vibrotactile sensations to a user’s body without contact,158,159) suggesting its potential integration with a typical projector in PM. Olfactory projection systems160,161) could also be integrated into PM, offering the potential to provide novel visual-olfactory experiences to users.
整合其他感官模态的投影系统(称为 X)到全彩 PM(即 RGB-X PM)以扩展用户体验是另一个有希望的研究方向。例如,研究人员将典型的 RGB 投影仪与能够改变远红外光点方向的加热投影仪相结合,同时为用户身体提供可见和热觉。此外,基于超声波相控阵的空中振动触觉显示器可以在不接触的情况下为用户身体提供振动触觉,暗示其与典型投影仪在 PM 中的潜在集成。嗅觉投影系统 160,161) 也可以集成到 PM 中,为用户提供新颖的视觉-嗅觉体验。
9. Conclusion 9. 结论
This review has introduced the current trends in the PM research from 2018 and later, as well as future directions. To recap, the notable trends include low-latency DPM, high quality radiometric compensation and deblurring by DNNs, delay-free shadow removal by large aperture optics, unconventional projectors for various tasks, and overcoming technical limitations by combining near-eye optics and perceptual tricks. Expected future directions encompass much lower-latency DPM, the pursuit of perceptual realism, and the development of RGB-X PM. The field of PM research will continue to evolve, integrating diverse disciplines such as computer science, optics, psychology, and electrical and electronic engineering.
本评论介绍了 2018 年及以后项目管理(PM)研究中的当前趋势以及未来方向。总结来说,值得注意的趋势包括低延迟的 DPM、通过深度神经网络(DNNs)实现的高质量辐射补偿和去模糊、通过大孔径光学实现的无延迟阴影消除、用于各种任务的非传统投影仪,以及通过结合近眼光学和感知技巧克服技术限制。预期的未来方向包括更低延迟的 DPM、追求感知真实感以及 RGB-X PM 的发展。项目管理研究领域将继续发展,整合计算机科学、光学、心理学以及电气和电子工程等众多学科。
Non-standard abbreviation list
非常规缩写列表
- AR AR (由于“AR”可能指代多种含义,如技术术语、品牌名称等,这里直接保留原样。)
augmented reality 增强现实
- DNN 深度神经网络
deep neural network 深度神经网络
- DPM DPM (注:DPM 可能是一个缩写,没有上下文无法确定其具体含义,因此直接保留原样。)
dynamic projection mapping
动态投影映射- ETL
electrically focus-tunable lens
电控焦距可调镜头- IR IR (注:"IR" 在此上下文中可能是一个缩写或专有名词,因此未进行翻译。)
infrared 红外
- PM PM 请注意,"PM" 通常指的是下午或晚上的时间,但在没有上下文的情况下,它也可以被视为一个缩写或专业术语。由于没有提供具体的上下文,我将保留原样
projection mapping 投影映射
Profile 个人资料
Daisuke Iwai was born in Ehime in 1980. After receiving his PhD degree from Osaka University in 2007, he started his academic career at Osaka University. He was also a visiting scientist at Bauhaus-University Weimar, Germany, from 2007 to 2008 and a visiting Associate Professor at ETH, Switzerland, in 2011. He is currently working as an Associate Professor at the Graduate School of Engineering Science, Osaka University, Japan. His research interests include augmented reality, projection mapping, and human–computer interaction. He is also currently serving as an Associate Editor of IEEE Transactions on Visualization and Computer Graphics and has previously served as Program Chairs of IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (2021, 2022) and IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (2022). His publications have received Best Paper Awards at IEEE VR (2015), IEEE Symposium on 3D User Interfaces (3DUI) (2015), and IEEE ISMAR (2021). He is a recipient of the JSPS Prize (2023).
伊豆井大介于 1980 年出生于爱媛县。2007 年从大阪大学获得博士学位后,他开始在 Osaka University 从事学术生涯。他还于 2007 年至 2008 年在德国魏玛包豪斯大学担任访问科学家,并在 2011 年担任瑞士 ETH 的访问副教授。他目前在日本大阪大学工程科学研究生院担任副教授。他的研究兴趣包括增强现实、投影映射和人与计算机交互。他目前还担任 IEEE Transactions on Visualization and Computer Graphics 的副编辑,并曾担任 IEEE 国际混合与增强现实研讨会(ISMAR)(2021,2022)和 IEEE 虚拟现实与 3D 用户界面会议(VR)(2022)的程序主席。他的出版物在 IEEE VR(2015)、IEEE 3D 用户界面研讨会(3DUI)(2015)和 IEEE ISMAR(2021)上获得了最佳论文奖。他是 2023 年 JSPS 奖获得者。
References 参考文献
-
1).Iwai D., Yabiki T., Sato K. (2013) View management of projected labels on nonplanar and textured surfaces. IEEE Trans. Vis. Comput. Graph.
19, 1415–1424.
[DOI] [PubMed] [Google Scholar]
1).Iwai D.,Yabiki T.,Sato K.(2013)非平面和纹理表面的投影标签视图管理。IEEE Trans. Vis. Comput. Graph. 19,1415-1424。[DOI] [PubMed] [Google Scholar] -
2).Siegl C., Colaianni M., Thies L., Thies J., Zollhöfer M., Izadi S., et al. (2015) Real-time pixel luminance optimization for dynamic multi-projection mapping. ACM Trans. Graph.
34, 237. [Google Scholar]
2).Siegl C.,Colaianni M.,Thies L.,Thies J.,Zollhöfer M.,Izadi S.,等(2015)动态多投影映射的实时像素亮度优化。ACM Trans. Graph. 34,237。[谷歌学术] -
3).Bimber, O. and Raskar, R. (2005) Spatial Augmented Reality: Merging Real and Virtual Worlds. A. K. Peters, Ltd., Wellesley, MA, USA. [Google Scholar]
3).Bimber, O. 和 Raskar, R. (2005) 空间增强现实:融合现实与虚拟世界。A. K. Peters, Ltd.,Wellesley,MA,美国。[谷歌学术] -
4).Jones, B.R., Benko, H., Ofek, E. and Wilson, A.D. (2013) Illumiroom: Peripheral projected illusions for interactive experiences. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13, Association for Computing Machinery, New York, NY, USA, pp. 869–878. [Google Scholar]
4).琼斯,B.R.,本科,H.,奥菲克,E.和威尔逊,A.D.(2013)Illumiroom:交互体验的边缘投影幻觉。在 SIGCHI 会议关于人机交互因素的论文集中,CHI '13,美国纽约,纽约,美国计算机协会,第 869-878 页。[谷歌学术] -
5).Mine M.R., van Baar J., Grundhofer A., Rose D., Yang B. (2012) Projection-based augmented reality in Disney theme parks. Computer
45, 32–40. [Google Scholar]
5).Mine M.R.,van Baar J.,Grundhofer A.,Rose D.,Yang B.(2012)基于投影的迪士尼主题公园增强现实。计算机 45,32-40。[谷歌学术] -
6).Wilson, A., Benko, H., Izadi, S. and Hilliges, O. (2012) Steerable augmented reality with the beamatron. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, UIST ’12, Association for Computing Machinery, New York, NY, USA, pp. 413–422. [Google Scholar]
6).威尔逊,A.,本科,H.,伊扎迪,S.和希尔吉斯,O.(2012)使用 beamatron 的可调节增强现实。在 25 届年度 ACM 用户界面软件和技术研讨会论文集,UIST '12,美国计算机协会,纽约,纽约,美国,第 413-422 页。[谷歌学术] -
7).Pinhanez, C. (2001) Using a steerable projector and a camera to transform surfaces into interactive displays. In CHI ’01 Extended Abstracts on Human Factors in Computing Systems, pp. 369–370. [Google Scholar]
7).Pinhanez, C. (2001) 使用可调向投影仪和相机将表面转换为交互式显示屏。在 CHI’01 关于人机交互系统因素的扩展摘要中,第 369-370 页。[谷歌学术] -
8).Iwai, D. and Sato, K. (2006) Limpid desk: See-through access to disorderly desktop in projection-based mixed reality. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST ’06, Association for Computing Machinery, New York, NY, USA, pp. 112–115. [Google Scholar]
8).岩井,D. 和 佐藤,K. (2006) 透明桌面:基于投影的混合现实中对杂乱桌面的透明访问。在 ACM 虚拟现实软件与技术研讨会论文集,VRST '06,美国纽约,纽约州,美国计算机协会,第 112-115 页。[谷歌学术] -
9).Iwai D., Sato K. (2011) Document search support by making physical documents transparent in projection-based mixed reality. Virtual Real.
15, 147–160. [Google Scholar]
9).岩井 D.,佐藤 K.(2011)通过投影混合现实使物理文档透明以支持文档搜索。虚拟现实. 15,147–160。[谷歌学术] -
10).Kitajima Y., Iwai D., Sato K. (2017) Simultaneous projection and positioning of laser projector pixels. IEEE Trans. Vis. Comput. Graph.
23, 2419–2429.
[DOI] [PubMed] [Google Scholar]
10).北岛洋一,岩井大辅,佐藤耕平(2017)激光投影仪像素的同时投影和定位。IEEE 可视计算与图形学杂志,23,2419-2429。[DOI] [PubMed] [Google Scholar] -
11).Matsushita, K., Iwai, D. and Sato, K. (2011) Interactive bookshelf surface for in situ book searching and storing support. In Proceedings of the 2nd Augmented Human International Conference (Tokyo, 2011), Article No. 2. [Google Scholar]
11).松下,K.,岩井,D.和佐藤,K.(2011)现场图书搜索和存储支持的交互式书架表面。在第二届增强人类国际会议(东京,2011 年)论文集,文章编号 2。[谷歌学术] -
12).Raskar R., Beardsley P., van Baar J., Wang Y., Dietz P., Lee J., et al. (2004) Rfig lamps: Interacting with a self-describing world via photosensing wireless tags and projectors. ACM Trans. Graph.
23, 406–415. [Google Scholar]
12).Raskar R.,Beardsley P.,van Baar J.,Wang Y.,Dietz P.,Lee J.,等.(2004)Rfig 灯具:通过光电感应无线标签和投影仪与自描述世界的交互。ACM Trans. Graph. 23,406–415。[谷歌学术] -
13).Nishino H., Hatano E., Seo S., Nitta T., Saito T., Nakamura M., et al. (2018) Real-time navigation for liver surgery using projection mapping with indocyanine green fluorescence: Development of the novel medical imaging projection system. Ann. Surg.
267, 1134–1140.
[DOI] [PubMed] [Google Scholar]
13).西野浩一,田中恵,瀬尾聡,新田貴史,斎藤哲也,中村雅夫等(2018)利用吲哚菁绿荧光投影导航的肝脏手术实时导航系统开发:新型医学影像投影系统的开发。外科学报 267,1134–1140. [ DOI] [ PubMed] [ Google Scholar] -
14).Bandyopadhyay, D., Raskar, R. and Fuchs, H. (2001) Dynamic shader lamps: painting on movable objects. In Proceedings IEEE and ACM International Symposium on Augmented Reality, pp. 207–216. [Google Scholar]
14).班迪奥帕达亚,D.,拉卡尔,R.和富克斯,H.(2001)动态着色灯:在可移动物体上绘画。在 IEEE 和 ACM 国际增强现实研讨会论文集中,第 207-216 页。[谷歌学术] -
15).Flagg, M. and Rehg, J.M. (2006) Projector-guided painting. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology, pp. 235–244. [Google Scholar]
15). Flagg, M. 和 Rehg, J.M. (2006) 投影引导绘画。在第 19 届 ACM 用户界面软件和技术年会上发表,第 235-244 页。[谷歌学术] -
16).Rivers A., Adams A., Durand F. (2012) Sculpting by numbers. ACM Trans. Graph.
31, 157. [Google Scholar]
16).Rivers A.,Adams A.,Durand F.(2012)数字雕塑。ACM Trans. Graph. 31,157。[谷歌学术] -
17).Bimber O., Coriand F., Kleppe A., Bruns E., Zollmann S., Langlotz T. (2005) Superimposing pictorial artwork with projected imagery. IEEE Multimed.
12, 16–26. [Google Scholar]
17).Bimber O.,Coriand F.,Kleppe A.,Bruns E.,Zollmann S.,Langlotz T.(2005)将图像艺术与投影图像叠加。IEEE 多媒体。12,16-26。[谷歌学术] -
18).Schmidt S., Bruder G., Steinicke F. (2019) Effects of virtual agent and object representation on experiencing exhibited artifacts. Comput. Graph.
83, 1–10. [Google Scholar]
18).施密特 S.,布鲁德 G.,施泰尼克 F.(2019)虚拟代理和物体表示对体验展品的影响。计算机图形学 83,1-10。[谷歌学术] -
19).Iwai D., Matsukage R., Aoyama S., Kikukawa T., Sato K. (2018) Geometrically consistent projection-based tabletop sharing for remote collaboration. IEEE Access
6, 6293–6302. [Google Scholar]
19).岩井 D.,松川 R.,青山 S.,菊川 T.,佐藤 K.(2018)基于几何一致投影的桌面共享远程协作。IEEE Access 6,6293–6302。[谷歌学术] -
20).Pejsa, T., Kantor, J., Benko, H., Ofek, E. and Wilson, A. (2016) Room2room: Enabling life-size telepresence in a projected augmented reality environment. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW ’16, pp. 1716–1725. [Google Scholar]
20).Pejsa, T.,Kantor, J.,Benko, H.,Ofek, E. 和 Wilson, A. (2016) Room2room:在投影增强现实环境中实现全尺寸远程呈现。在 19 届 ACM 计算机支持的协作工作和社交计算会议论文集中,CSCW '16,第 1716-1725 页。[谷歌学术] -
21).Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L. and Fuchs, H. (1998) The office of the future: A unified approach to image-based modeling and spatially immersive displays. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’98, pp. 179–188. [Google Scholar]
21).Raskar, R.,Welch,G.,Cutts,M.,Lake,A.,Stesin,L.和 Fuchs,H.(1998)未来办公室:基于图像建模和空间沉浸式显示的统一方法。在计算机图形学与交互技术第 25 届年度会议论文集,SIGGRAPH’98,第 179-188 页。[谷歌学术] -
22).Duan, T., Punpongsanon, P., Iwai, D. and Sato, K. (2018) Flyinghand: Extending the range of haptic feedback on virtual hand using drone-based object recognition. In SIGGRAPH Asia 2018 Technical Briefs, SA ’18, Association for Computing Machinery, New York, NY, USA, Article No. 28. [Google Scholar]
22).段,T., Punpongsanon,P.,Iwai,D.和 Sato,K.(2018)Flyinghand:利用基于无人机的物体识别扩展虚拟手触觉反馈的范围。在 SIGGRAPH 亚洲 2018 技术简报,SA’18,美国纽约,纽约,美国计算机协会,文章编号 28。[谷歌学术] -
23).Ueda, Y., Asai, Y., Enomoto, R., Wang, K., Iwai, D. and Sato, K. (2017) Body cyberization by spatial augmented reality for reaching unreachable world. In Proceedings of the 8th Augmented Human International Conference, AH ’17, Association for Computing Machinery, New York, NY, USA, Article No. 19. [Google Scholar]
23).上田,Y.,浅井,Y.,榎本,R.,王,K.,岩井,D.和佐藤,K.(2017)通过空间增强现实实现身体网络化以触及无法触及的世界。在第八届增强人类国际会议论文集,AH’17,美国纽约,纽约,美国计算机协会,文章编号 19。[谷歌学术] -
24).Xu, H., Iwai, D., Hiura, S. and Sato, K. (2006) User interface by virtual shadow projection. In 2006 SICE-ICASE International Joint Conference, pp. 4814–4817. [Google Scholar]
24). 徐 H,岩井 D,广浦 S,佐藤 K.(2006)虚拟阴影投影用户界面. 在 2006 年 SICE-ICASE 国际联合会议,第 4814-4817 页。[谷歌学术] -
25).Bermano A.H., Billeter M., Iwai D., Grundhöfer A. (2017) Makeup lamps: Live augmentation of human faces via projection. Comput. Graph. Forum
36, 311–323. [Google Scholar]
25).Bermano A.H.,Billeter M.,Iwai D.,Grundhöfer A.(2017)化妆灯:通过投影实现的人脸实时增强。计算机图形学论坛 36,311–323。[谷歌学术] -
26).Siegl C., Lange V., Stamminger M., Bauer F., Thies J. (2017) Faceforge: Markerless non-rigid face multi-projection mapping. IEEE Trans. Vis. Comput. Graph.
23, 2440–2446.
[DOI] [PubMed] [Google Scholar]
26).Siegl C.,Lange V.,Stamminger M.,Bauer F.,Thies J.(2017)Faceforge:无标记的非刚性人脸多投影映射。IEEE Trans. Vis. Comput. Graph. 23,2440–2446。[DOI] [PubMed] [Google Scholar] -
27).Cascini G., O’Hare J., Dekoninck E., Becattini N., Boujut J.-F., Ben Guefrache F., et al. (2020) Exploring the use of ar technology for co-creative product and packaging design. Comput. Ind.
123, 103308. [Google Scholar]
27).Cascini G.,O’Hare J.,Dekoninck E.,Becattini N.,Boujut J.-F.,Ben Guefrache F.,等.(2020)探索增强现实技术在协同创意产品与包装设计中的应用. Comput. Ind. 123,103308. [谷歌学术] -
28).Marner M.R., Smith R.T., Walsh J.A., Thomas B.H. (2014) Spatial user interfaces for large-scale projector-based augmented reality. IEEE Comput. Graph. Appl.
34, 74–82.
[DOI] [PubMed] [Google Scholar]
28).Marner M.R.,Smith R.T.,Walsh J.A.,Thomas B.H.(2014)基于大规模投影增强现实的空间用户界面。IEEE 计算机图形学应用 34,74-82。[DOI] [PubMed] [谷歌学术] -
29).Menk C., Jundt E., Koch R. (2011) Visualisation techniques for using spatial augmented reality in the design process of a car. Comput. Graph. Forum
30, 2354–2366. [Google Scholar]
29).Menk C.,Jundt E.,Koch R.(2011)在汽车设计过程中的空间增强现实可视化技术。Comput. Graph. Forum 30,2354–2366。[谷歌学术] -
30).Takezawa, T., Iwai, D., Sato, K., Hara, T., Takeda, Y. and Murase, K. (2019) Material surface reproduction and perceptual deformation with projection mapping for car interior design. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 251–258. [Google Scholar]
30).竹沢,T.,岩井,D.,佐藤,K.,原,T.,竹田,Y. 和村瀬,K. (2019) 基于投影映射的汽车内饰设计中的材料表面再现和感知变形。在 2019 年 IEEE 虚拟现实与 3D 用户界面会议(VR),第 251-258 页。[谷歌学术] -
31).Bimber O., Iwai D., Wetzstein G., Grundhöfer A. (2008) The visual computing of projector-camera systems. Comput. Graph. Forum
27, 2219–2245. [Google Scholar]
31).Bimber O.,Iwai D.,Wetzstein G.,Grundhöfer A.(2008)投影仪-摄像机系统的视觉计算. 计算机图形学论坛 27,2219-2245. [谷歌学术] -
32).Grundhöfer A., Iwai D. (2018) Recent advances in projection mapping algorithms, hardware and applications. Comput. Graph. Forum
37, 653–675. [Google Scholar]
32). Grundhöfer A., Iwai D. (2018) 投影映射算法、硬件和应用方面的最新进展。计算机图形学论坛 37,653–675。[谷歌学术] -
33).Zhang Z. (2000) A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell.
22, 1330–1334. [Google Scholar]
33). 张 Z. (2000) 一种灵活的相机标定新方法。IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334. [谷歌学术] -
34).Xie, C., Shishido, H., Kameda, Y. and Kitahara, I. (2019) A projector calibration method using a mobile camera for projection mapping system. In 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 261–262. [Google Scholar]
34).谢,C.,志田,H.,加藤,Y.和北原,I.(2019)使用移动相机进行投影映射系统的投影校准方法。在 2019 年 IEEE 国际混合与增强现实研讨会附属会议(ISMAR-Adjunct),第 261-262 页。[谷歌学术] -
35).Sugimoto M., Iwai D., Ishida K., Punpongsanon P., Sato K. (2021) Directionally decomposing structured light for projector calibration. IEEE Trans. Vis. Comput. Graph.
27, 4161–4170.
[DOI] [PubMed] [Google Scholar]
35).Sugimoto M.,Iwai D.,Ishida K.,Punpongsanon P.,Sato K.(2021)投影仪校准中结构光的方向分解。IEEE Trans. Vis. Comput. Graph. 27,4161–4170。[DOI] [PubMed] [Google Scholar] -
36).Yoon, D., Kim, J., Jo, J. and Kim, K. (2023) Projection mapping method using projector-lidar (light detection and ranging) calibration. In 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 857–858. [Google Scholar]
36).Yoon, D.,Kim, J.,Jo, J. 和 Kim, K.(2023)使用投影仪激光雷达(光探测与测距)校准的投影映射方法。在 2023 年 IEEE 虚拟现实与 3D 用户界面会议摘要和工作坊(VRW),第 857-858 页。[谷歌学术] -
37).Kurth P., Lange V., Siegl C., Stamminger M., Bauer F. (2018) Auto-calibration for dynamic multi-projection mapping on arbitrary surfaces. IEEE Trans. Vis. Comput. Graph.
24, 2886–2894.
[DOI] [PubMed] [Google Scholar]
37).Kurth P.,Lange V.,Siegl C.,Stamminger M.,Bauer F.(2018)任意表面的动态多投影映射的自校准。IEEE Trans. Vis. Comput. Graph. 24,2886-2894。[DOI] [PubMed] [Google Scholar] -
38).Tehrani M.A., Gopi M., Majumder A. (2021) Automated geometric registration for multi-projector displays on arbitrary 3D shapes using uncalibrated devices. IEEE Trans. Vis. Comput. Graph.
27, 2265–2279.
[DOI] [PubMed] [Google Scholar]
38).Tehrani M.A.,Gopi M.,Majumder A.(2021)使用未校准设备在任意 3D 形状上的多投影仪显示的自动几何配准。IEEE Trans. Vis. Comput. Graph. 27,2265-2279。[DOI] [PubMed] [Google Scholar] -
39).Ueno, A., Amano, T. and Yamauchi, C. (2022) Geometric calibration with multi-viewpoints for multi-projector systems on arbitrary shapes using homography and pixel maps. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 828–829. [Google Scholar]
39).上野 A,天野 T,山内 C.(2022)基于单应性和像素映射的任意形状多投影系统多视角几何标定. 在 2022 年 IEEE 虚拟现实与 3D 用户界面会议摘要和工作坊(VRW),第 828-829 页。[谷歌学术] -
40).Ahmed B., Lee K.H. (2019) Projection mapping onto deformable nonrigid surfaces using adaptive selection of fiducials. J. Electron. Imaging
28, 063008. [Google Scholar]
40).Ahmed B.,李 K.H.(2019)基于自适应选择标记点的可变形非刚性表面投影映射。电子成像杂志 28,063008。[谷歌学术] -
41).Ibrahim, M.T., Gopi, M. and Majumder, A. (2023) Self-calibrating dynamic projection mapping system for dynamic, deformable surfaces with jitter correction and occlusion handling. In 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 293–302. [Google Scholar]
41).伊布拉希姆,M.T.,戈皮,M.,马朱默德,A.(2023)具有抖动校正和遮挡处理的动态、可变形表面的自校准动态投影映射系统。在 2023 年 IEEE 国际混合与增强现实研讨会(ISMAR),第 293-302 页。[谷歌学术] -
42).Ibrahim, M.T., Meenakshisundaram, G. and Majumder, A. (2020) Dynamic projection mapping of deformable stretchable materials. In Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology, VRST ’20, Association for Computing Machinery, New York, NY, USA, Article No. 35. [Google Scholar]
42).伊布拉希姆,M.T.,梅纳克希桑达拉姆,G.和马朱姆德,A.(2020)可变形可拉伸材料的动态投影映射。在 26 届 ACM 虚拟现实软件与技术研讨会论文集,VRST '20,美国纽约,纽约,美国计算机协会,文章编号 35。[谷歌学术] -
43).Miyazaki, D. and Hashimoto, N. (2018) Dynamic projection mapping onto non-rigid objects with dot markers. In 2018 International Workshop on Advanced Image Technology (IWAIT), doi: 10.1109/IWAIT.2018.8369679. [Google Scholar]
43).宫崎,D. 和橋本,N. (2018) 基于点标记的动态投影映射到非刚性物体。在 2018 年国际先进图像技术研讨会(IWAIT),doi: 10.1109/IWAIT.2018.8369679. [谷歌学术] -
44).Maeda, K. and Koike, H. (2020) MirAIProjection: Real-time projection onto high-speed objects by predicting their 3D position and pose using DNNs. In Proceedings of the International Conference on Advanced Visual Interfaces, AVI ’20, Association for Computing Machinery, New York, NY, USA, Article No. 59. [Google Scholar]
44).前田耕平,小池浩(2020)MirAIProjection:通过预测高速物体的 3D 位置和姿态使用深度神经网络进行实时投影。在《国际高级视觉界面会议》论文集中,AVI '20,美国纽约,纽约州,美国计算机协会,文章编号 59。[谷歌学术] -
45).Lee K., Sim K., Uhm T., Lee S.H., Park J.-I. (2020) Design of imperceptible metamer markers and application systems. IEEE Access
8, 53687–53696. [Google Scholar]
45).李 K.,辛 K.,乌姆 T.,李 S.H.,朴 J.-I.(2020)不可察觉的元 amer 标记及其应用系统设计。IEEE Access 8,53687–53696。[谷歌学术] -
46).Tone D., Iwai D., Hiura S., Sato K. (2020) Fibar: Embedding optical fibers in 3D printed objects for active markers in dynamic projection mapping. IEEE Trans. Vis. Comput. Graph.
26, 2030–2040.
[DOI] [PubMed] [Google Scholar]
46).Tone D.,Iwai D.,Hiura S.,Sato K.(2020)Fibar:将光纤嵌入 3D 打印物体以用于动态投影映射的主动标记。IEEE Trans. Vis. Comput. Graph. 26,2030-2040。[DOI] [PubMed] [Google Scholar] -
47).Asayama H., Iwai D., Sato K. (2018) Fabricating diminishable visual markers for geometric registration in projection mapping. IEEE Trans. Vis. Comput. Graph.
24, 1091–1102.
[DOI] [PubMed] [Google Scholar]
47).Asayama H., Iwai D., Sato K. (2018) 投影映射中用于几何配准的可消除视觉标记的制造。IEEE 计算机视觉与图形学杂志 24,1091–1102。[DOI] [PubMed] [Google Scholar] -
48).Oku, H., Nomura, M., Shibahara, K. and Obara, A. (2018) Edible projection mapping. In SIGGRAPH Asia 2018 Emerging Technologies, SA ’18, Association for Computing Machinery, New York, NY, USA, Article No. 2. [Google Scholar]
48).Oku, H.,野村,M.,柴原,K. 和 大原,A. (2018) 可食用投影映射。在 SIGGRAPH 亚洲 2018 新兴技术,SA’18,美国计算机协会,纽约,纽约州,美国,文章编号 2。[谷歌学术] -
49).Miyatake, Y., Punpongsanon, P., Iwai, D. and Sato, K. (2022) interiqr: Unobtrusive edible tags using food 3D printing. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST ’22, Association for Computing Machinery, New York, NY, USA, Article No. 84. [Google Scholar]
49).宫畑,Y.,潘蓬蓬,P.,岩井,D.和佐藤,K.(2022)interiqr:利用食品 3D 打印的无干扰可食用标签。在 35 届年度 ACM 用户界面软件和技术研讨会论文集,UIST '22,美国纽约,纽约州,美国计算机协会,文章编号 84。[谷歌学术] -
50).Halvorson, Y., Saito, T. and Hashimoto, N. (2022) Robust tangible projection mapping with multi-view contour-based object tracking. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 756–757. [Google Scholar]
50).Halvorson, Y., Saito, T. 和 Hashimoto, N. (2022) 基于多视图轮廓的鲁棒触觉投影映射与物体跟踪。在 2022 年 IEEE 虚拟现实与 3D 用户界面会议摘要和工作坊(VRW),第 756-757 页。[谷歌学术] -
51).Kurth P., Leuschner M., Stamminger M., Bauer F. (2022) Content-aware brightness solving and error mitigation in large-scale multi-projection mapping. IEEE Trans. Vis. Comput. Graph.
28, 3607–3617.
[DOI] [PubMed] [Google Scholar]
51).Kurth P.,Leuschner M.,Stamminger M.,Bauer F.(2022)大规模多投影映射中的内容感知亮度解决和错误缓解。IEEE Trans. Vis. Comput. Graph. 28,3607-3617。[DOI] [PubMed] [Google Scholar] -
52).Ng, A., Lepinski, J., Wigdor, D., Sanders, S. and Dietz, P. (2012) Designing for low-latency direct-touch input. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, UIST ’12, Association for Computing Machinery, New York, NY, USA, pp. 453–464. [Google Scholar]
52).Ng, A.,Lepinski, J.,Wigdor, D.,Sanders, S. 和 Dietz, P. (2012) 低延迟直接触摸输入的设计。在 25 届年度 ACM 用户界面软件和技术研讨会论文集,UIST '12,美国纽约,纽约州,美国计算机协会,第 453-464 页。[谷歌学术] -
53).Watanabe, Y. and Ishikawa, M. (2019) High-speed and high-brightness color single-chip dlp projector using high-power led-based light sources. In 26th International Display Workshops, IDW 2019, pp. 1350–1352. [Google Scholar]
53).渡辺,Y. 和石川,M.(2019)基于高功率 LED 光源的高速高亮度单芯片 DLP 投影仪。在第 26 届国际显示研讨会,IDW 2019,第 1350-1352 页。[谷歌学术] -
54).Mikawa, Y., Sueishi, T., Watanabe, Y. and Ishikawa, M. (2018) Variolight: Hybrid dynamic projection mapping using high-speed projector and optical axis controller. In SIGGRAPH Asia 2018 Emerging Technologies, SA ’18, Association for Computing Machinery, New York, NY, USA, Article No. 17. [Google Scholar]
54).Mikawa, Y., Sueishi, T., Watanabe, Y. 和 Ishikawa, M. (2018) Variolight:使用高速投影仪和光学轴控制器的高动态投影映射。在 SIGGRAPH 亚洲 2018 新兴技术,SA’18,美国纽约,纽约,美国,文章编号 17。[谷歌学术] -
55).Mikawa Y., Sueishi T., Watanabe Y., Ishikawa M. (2022) Dynamic projection mapping for robust sphere posture tracking using uniform/biased circumferential markers. IEEE Trans. Vis. Comput. Graph.
28, 4016–4031.
[DOI] [PubMed] [Google Scholar]
55).Mikawa Y.,Sueishi T.,Watanabe Y.,Ishikawa M.(2022)基于均匀/偏心圆周标记的动态投影映射实现鲁棒的球体姿态跟踪。IEEE Trans. Vis. Comput. Graph. 28,4016-4031。[DOI] [PubMed] [Google Scholar] -
56).Sueishi, T. and Ishikawa, M. (2021) Ellipses ring marker for high-speed finger tracking. In Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology, VRST ’21, Association for Computing Machinery, New York, NY, USA, Article No. 31. [Google Scholar]
56).Sueishi, T. 和 Ishikawa, M. (2021) 高速手指跟踪的省略号环形标记。在 27 届 ACM 虚拟现实软件与技术研讨会论文集,VRST '21,美国纽约,纽约,纽约,美国,文章编号 31。[谷歌学术] -
57).Peng H.-L., Watanabe Y. (2021) High-speed dynamic projection mapping onto human arm with realistic skin deformation. Appl. Sci.
11, 3753. [Google Scholar]
57).彭 H.-L.,渡边 Y.(2021)高速动态投影映射到具有真实皮肤变形的人臂。应用科学 11,3753。[谷歌学术] -
58).Nomoto T., Li W., Peng H.-L., Watanabe Y. (2022) Dynamic multi-projection mapping based on parallel intensity control. IEEE Trans. Vis. Comput. Graph.
28, 2125–2134.
[DOI] [PubMed] [Google Scholar]
58).Nomoto T.,李伟,彭慧玲,渡边洋(2022)基于并行强度控制的动态多投影映射。IEEE Transactions on Visualization and Computer Graphics 28,2125–2134。[DOI] [PubMed] [Google Scholar] -
59).Nomoto, T., Koishihara, R. and Watanabe, Y. (2020) Realistic dynamic projection mapping using real-time ray tracing. In ACM SIGGRAPH 2020 Emerging Technologies, SIGGRAPH ’20, Association for Computing Machinery, New York, NY, USA, Article No. 13. [Google Scholar]
59).Nomoto, T.,小石原,R. 和渡边,Y.(2020)使用实时光线追踪的逼真动态投影映射。在 ACM SIGGRAPH 2020 新兴技术,SIGGRAPH '20,美国计算机协会,纽约,纽约,美国,文章编号 13。[谷歌学术] -
60).Tsurumi, N., Ohishi, K., Kakimoto, R., Tsukiyama, F., Peng, H.-L., Watanabe, Y. et al. (2023) Rediscovering your own beauty through a highly realistic 3D digital makeup system based on projection mapping technology. In 33rd IFSCC Congress, RB-03. [Google Scholar]
60).Tsurumi, N.,Ohishi, K.,Kakimoto, R.,Tsukiyama, F.,Peng, H.-L.,Watanabe, Y. 等人(2023)通过基于投影映射技术的超逼真 3D 数字化妆系统重新发现你的美丽。在第 33 届 IFSCC 大会上,RB-03。[谷歌学术] -
61).Kagami S., Hashimoto K. (2019) Animated stickies: Fast video projection mapping onto a markerless plane through a direct closed-loop alignment. IEEE Trans. Vis. Comput. Graph.
25, 3094–3104.
[DOI] [PubMed] [Google Scholar]
61).Kagami S.,Hashimoto K.(2019)动画便签:通过直接闭环对齐快速视频投影映射到无标记平面上。IEEE Trans. Vis. Comput. Graph. 25,3094-3104。[DOI] [PubMed] [Google Scholar] -
62).Hisaichi, S., Sumino, K., Ueda, K., Kasebe, H., Yamashita, T., Yuasa, T. et al. (2021) Depth-aware dynamic projection mapping using high-speed rgb and ir projectors. In SIGGRAPH Asia 2021 Emerging Technologies, SA ’21, Association for Computing Machinery, New York, NY, USA, Article No. 3. [Google Scholar]
62).Hisaichi, S.,Sumino, K.,Ueda, K.,Kasebe, H.,Yamashita, T.,Yuasa, T. 等人(2021)使用高速 RGB 和红外投影仪的深度感知动态投影映射。在 SIGGRAPH 亚洲 2021 新兴技术,SA’21,美国纽约,纽约,纽约,美国,文章编号 3。[谷歌学术] -
63).Miyashita L., Watanabe Y., Ishikawa M. (2018) Midas projection: Markerless and modelless dynamic projection mapping for material representation. ACM Trans. Graph.
37, 196. [Google Scholar]
63).宫下 L.,渡边 Y.,石川 M.(2018)Midas 投影:用于材料表示的无标记和无模型动态投影映射。ACM 图形学传输。37,196。[谷歌学术] -
64).Iuchi, M., Hirohashi, Y. and Oku, H. (2023) Proposal for an aerial display using dynamic projection mapping on a distant flying screen. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR), pp. 603–608. [Google Scholar]
64).Iuchi, M.,Hirohashi, Y. 和 Oku, H. (2023) 关于在远距离飞行屏幕上使用动态投影映射进行空中展示的提案。载于 2023 年 IEEE 虚拟现实与 3D 用户界面会议(VR),第 603-608 页。[谷歌学术] -
65).Miyashita, L., Yamazaki, T., Uehara, K., Watanabe, Y. and Ishikawa, M. (2018) Portable Lumipen: Dynamic sar in your hand. In 2018 IEEE International Conference on Multimedia and Expo (ICME), doi: 10.1109/ICME.2018.8486514. [Google Scholar]
65).宫下,L.,山崎,T.,上原,K.,渡边,Y. 和石川,M. (2018) 可携式 Lumipen:手边的动态 SAR。在 2018 年 IEEE 国际多媒体与展览会议(ICME),doi: 10.1109/ICME.2018.8486514. [谷歌学术] -
66).Post, M., Fieguth, P., Naiel, M.A., Azimifar, Z. and Lamm, M. (2019) Fresco: Fast radiometric egocentric screen compensation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1899–1906. [Google Scholar]
66).Post, M.,Fieguth, P.,Naiel, M.A.,Azimifar, Z. 和 Lamm, M. (2019) Fresco:快速辐射自中心屏幕补偿。在 2019 年 IEEE/CVF 计算机视觉与模式识别会议(CVPRW) workshops,第 1899-1906 页。[谷歌学术] -
67).Grundhöfer, A. (2013) Practical non-linear photometric projector compensation. In 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 924–929. [Google Scholar]
67).Grundhöfer, A. (2013) 实用非线性光度投影补偿。在 2013 年 IEEE 计算机视觉与模式识别研讨会,第 924-929 页。[谷歌学术] -
68).Grundhöfer A., Iwai D. (2015) Robust, error-tolerant photometric projector compensation. IEEE Trans. Image Process.
24, 5086–5099.
[DOI] [PubMed] [Google Scholar]
68).Grundhöfer A., Iwai D. (2015) 鲁棒的、容错的光度投影补偿。IEEE 图像处理杂志. 24, 5086–5099. [DOI] [PubMed] [谷歌学术] -
69).Kurth, P., Lange, V., Stamminger, M. and Bauer, F. (2020) Real-time adaptive color correction in dynamic projection mapping. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 174–184. [Google Scholar]
69).Kurth, P.,Lange, V.,Stamminger, M. 和 Bauer, F. (2020) 动态投影映射中的实时自适应色彩校正。在 2020 年 IEEE 国际混合与增强现实研讨会(ISMAR),第 174-184 页。[谷歌学术] -
70).Li Y., Majumder A., Gopi M., Wang C., Zhao J. (2018) Practical radiometric compensation for projection display on textured surfaces using a multidimensional model. Comput. Graph. Forum
37, 365–375. [Google Scholar]
70).李毅,马占德,高皮,王晨,赵杰(2018)基于多维模型的纹理表面投影显示的实用辐射补偿。计算机图形学论坛 37,365–375。[谷歌学术] -
71).Akiyama R., Yamamoto G., Amano T., Taketomi T., Plopski A., Sandor C., et al. (2021) Robust reflectance estimation for projection-based appearance control in a dynamic light environment. IEEE Trans. Vis. Comput. Graph.
27, 2041–2055.
[DOI] [PubMed] [Google Scholar]
71). akiyama R.,yamamoto G.,anamoto T.,taketomi T.,plopski A.,sandor C.,等.(2021)动态光照环境下基于投影的外观控制稳健反射率估计。IEEE Trans. Vis. Comput. Graph. 27,2041–2055。[DOI] [PubMed] [Google Scholar] -
72).Nishizawa, M. and Okajima, K. (2018) Precise surface color estimation using a non-diagonal reflectance matrix on an adaptive projector-camera system. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 190–195. [Google Scholar]
72).西泽,M. 和 岩治,K. (2018) 在自适应投影相机系统中使用非对角反射率矩阵进行精确表面颜色估计。载于 2018 年 IEEE 国际混合与增强现实研讨会附属会议(ISMAR-Adjunct),第 190-195 页。[谷歌学术] -
73).Amano, T. and Yoshioka, H. (2020) Viewing-direction dependent appearance manipulation based on light-field feedback. In Virtual Reality and Augmented Reality (eds. Bourdot, P., Interrante, V., Kopper, R., Olivier, A.-H., Saito, H. and Zachmann, G.). Springer International Publishing, Cham, pp. 192–205. [Google Scholar]
73).阿马诺,T. 和 吉冈,H. (2020) 基于光场反馈的视向依赖外观操控。在《虚拟现实与增强现实》(编者:Bourdot, P.,Interrante, V.,Kopper, R.,Olivier, A.-H.,Saito, H. 和 Zachmann, G.)。Springer 国际出版社,Cham,第 192-205 页。[谷歌学术] -
74).Kanaya, J. and Amano, T. (2022) Apparent shape manipulation by light-field projection onto a retroreflective surface. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 880–881. [Google Scholar]
74).Kanaya, J. 和 Amano, T. (2022) 通过光场投影到反光表面上的显形形状操纵。在 2022 年 IEEE 虚拟现实和 3D 用户界面会议摘要和工作坊(VRW)中,第 880-881 页。[谷歌学术] -
75).Kimura, K. and Amano, T. (2022) Perceptual brdf manipulation by 4-degree of freedom light field projection using multiple mirrors and projectors. In 2022 EuroXR Conference, pp. 95–99. [Google Scholar]
75).金丸,K. 和天野,T.(2022)使用多个镜子和投影仪的 4 自由度光场投影进行感知 BRDF 操作。在 2022 年 EuroXR 会议,第 95-99 页。[谷歌学术] -
76).Murakami, K. and Amano, T. (2018) Materiality manipulation by light-field projection from reflectance analysis. In ICAT-EGVE 2018 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (eds. Bruder, G., Yoshimoto, S. and Cobb, S.). The Eurographics Association, pp. 99–105. [Google Scholar]
76).村上,K. 和天野,T. (2018) 通过反射分析的光场投影的物质性操纵。在 2018 年国际人工智能现实与遥在会议和虚拟环境欧几里得研讨会(编辑:布鲁德,G.,吉本,S. 和科布,S.)。欧洲图形协会,第 99-105 页。[谷歌学术] -
77).Hashimoto N., Yoshimura K. (2021) Radiometric compensation for non-rigid surfaces by continuously estimating inter-pixel correspondence. Vis. Comput.
37, 175–187. [Google Scholar]
77).Hashimoto N.,Yoshimura K.(2021)通过连续估计像素间对应关系对非刚性表面的辐射补偿。Vis. Comput. 37,175–187。[谷歌学术] -
78).Pjanic P., Willi S., Iwai D., Grundhöfer A. (2018) Seamless multi-projection revisited. IEEE Trans. Vis. Comput. Graph.
24, 2963–2973. [DOI] [PubMed] [Google Scholar]
78).Pjanic P.,Willi S.,Iwai D.,Grundhöfer A.(2018)无缝多投影重访。IEEE Trans. Vis. Comput. Graph. 24,2963-2973。[DOI] [PubMed] [Google Scholar] -
79).Huang, B. and Ling, H. (2019) End-to-end projector photometric compensation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6803–6812. [Google Scholar]
79). 黄 B. 和 郎 H. (2019) 端到端投影仪光度补偿。在 2019 年 IEEE/CVF 计算机视觉与模式识别会议(CVPR),第 6803-6812 页。[谷歌学术] -
80).Huang, B. and Ling, H. (2019) Compennet++: End-to-end full projector compensation. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7164–7173. [Google Scholar]
80). 黄 B. 和 郎 H. (2019) Compennet++:端到端全投影补偿。在 2019 年 IEEE/CVF 国际计算机视觉会议(ICCV),第 7164-7173 页。[谷歌学术] -
81).Huang B., Sun T., Ling H. (2022) End-to-end full projector compensation. IEEE Trans. Pattern Anal. Mach. Intell.
44, 2953–2967.
[DOI] [PubMed] [Google Scholar]
81).黄 B.,孙 T.,凌 H.(2022)端到端全投影补偿。IEEE 模式分析与机器智能汇刊,44,2953–2967。[DOI] [PubMed] [谷歌学术] -
82).Park J., Jung D., Moon B. (2022) Projector compensation framework using differentiable rendering. IEEE Access
10, 44461–44470. [Google Scholar]
82).Park J., Jung D., Moon B. (2022) 基于可微分渲染的投影补偿框架。IEEE Access 10,44461-44470。[谷歌学术] -
83).Wang, Y., Ling, H. and Huang, B. (2023) Compenhr: Efficient full compensation for high-resolution projector. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR), pp. 135–145. [Google Scholar]
83).王,Y.,凌,H.和黄,B.(2023)Compenhr:高效的高分辨率投影仪全补偿。在 2023 年 IEEE 虚拟现实与 3D 用户界面会议(VR),第 135-145 页。[谷歌学术] -
84).Li Y., Yin W., Li J., Xie X. (2023) Physics-based efficient full projector compensation using only natural images. IEEE Trans. Vis. Comput. Graph.
doi: 10.1109/TVCG.2023.3281681. [DOI] [PubMed] [Google Scholar]
84).李毅,尹伟,李杰,谢晓东(2023)仅使用自然图像的基于物理的高效全投影补偿。IEEE Trans. Vis. Comput. Graph. doi: 10.1109/TVCG.2023.3281681. [ DOI] [ PubMed] [ Google Scholar] -
85).Huang B., Ling H. (2021) Deprocams: Simultaneous relighting, compensation and shape reconstruction for projector-camera systems. IEEE Trans. Vis. Comput. Graph.
27, 2725–2735.
[DOI] [PubMed] [Google Scholar]
85).黄 B., 郎 H. (2021) Deprocams:投影仪-摄像机系统的同时重光照、补偿和形状重建。IEEE Trans. Vis. Comput. Graph. 27, 2725–2735. [DOI] [PubMed] [Google Scholar] -
86).Erel Y., Iwai D., Bermano A.H. (2023) Neural projection mapping using reflectance fields. IEEE Trans. Vis. Comput. Graph.
29, 4339–4349.
[DOI] [PubMed] [Google Scholar]
86).Erel Y., Iwai D., Bermano A.H. (2023) 使用反射场进行神经投影映射。IEEE Trans. Vis. Comput. Graph. 29, 4339–4349. [DOI] [PubMed] [Google Scholar] -
87).Zhang, L. and Nayar, S. (2006) Projection defocus analysis for scene capture and image display. In ACM SIGGRAPH 2006 Papers, SIGGRAPH ’06, Association for Computing Machinery, New York, NY, USA, pp. 907–915. [Google Scholar]
87). 张,L. 和 纳亚尔,S. (2006) 场景捕捉和图像显示的投影失焦分析。在 ACM SIGGRAPH 2006 论文集,SIGGRAPH '06,美国计算机协会,纽约,纽约,美国,第 907-915 页。[谷歌学术] -
88).He Z., Li P., Zhao X., Zhang S., Tan J. (2021) Fast projection defocus correction for multiple projection surface types. IEEE Trans. Industr. Inform.
17, 3044–3055. [Google Scholar]
88).He Z., Li P., Zhao X., Zhang S., Tan J. (2021) 多投影表面类型的快速投影散焦校正。IEEE 工业信息传输 17,3044–3055。[谷歌学术] -
89).Kageyama Y., Isogawa M., Iwai D., Sato K. (2020) Prodebnet: projector deblurring using a convolutional neural network. Opt. Express
28, 20391–20403.
[DOI] [PubMed] [Google Scholar]
89).加藤洋一,伊贺川真,岩井大辅,佐藤康(2020)Prodebnet:使用卷积神经网络进行投影仪去模糊。光学快报 28,20391–20403。[DOI] [PubMed] [谷歌学术] -
90).Kageyama Y., Iwai D., Sato K. (2022) Online projector deblurring using a convolutional neural network. IEEE Trans. Vis. Comput. Graph.
28, 2223–2233.
[DOI] [PubMed] [Google Scholar]
90).加藤洋一,岩井大,佐藤耕平(2022)基于卷积神经网络的在线投影仪去模糊。IEEE 可视计算与图形学杂志,第 28 卷,2223-2233 页。[DOI] [PubMed] [谷歌学术] -
91).Xu H., Wang L., Tabata S., Watanabe Y., Ishikawa M. (2021) Extended depth-of-field projection method using a high-speed projectorwith a synchronized oscillating variable-focus lens. Appl. Opt.
60, 3917–3924.
[DOI] [PubMed] [Google Scholar]
91).徐 H.,王 L.,田畑 S.,渡辺 Y.,石川 M.(2021)使用同步振荡可变焦距镜头的高速投影仪扩展景深投影方法。应用光学 60,3917-3924。[DOI] [PubMed] [谷歌学术] -
92).Wang L., Tabata S., Xu H., Hu Y., Watanabe Y., Ishikawa M. (2023) Dynamic depth-of-field projection mapping method based on a variable focus lens and visual feedback. Opt. Express
31, 3945–3953.
[DOI] [PubMed] [Google Scholar]
92).王 L.,田畑 S.,徐 H.,胡 Y.,渡辺 Y.,石川 M.(2023)基于可变焦距镜头和视觉反馈的动态景深投影映射方法。光学快报 31,3945–3953。[DOI] [PubMed] [谷歌学术] -
93).Iwai D., Izawa H., Kashima K., Ueda T., Sato K. (2019) Speeded-up focus control of electrically tunable lens by sparse optimization. Sci. Rep.
9, 12365.
[DOI] [PMC free article] [PubMed] [Google Scholar]
93).Iwai D.,Izawa H.,Kashima K.,Ueda T.,Sato K.(2019)通过稀疏优化加速电调焦透镜的焦点控制。Sci. Rep. 9,12365。[DOI] [PMC 免费文章] [PubMed] [Google Scholar] -
94).Li, Y., Fu, Q. and Heidrich, W. (2023) Extended depth-of-field projector using learned diffractive optics. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR), pp. 449–459. [Google Scholar]
94).李,Y.,傅,Q.和海德里希,W.(2023)基于学习衍射光学扩展景深投影仪。在 2023 年 IEEE 虚拟现实与 3D 用户界面会议(VR),第 449-459 页。[谷歌学术] -
95).Ueda F., Kageyama Y., Iwai D., Sato K. (2023) Focal surface projection: Extending projector depth of field using a phase-only spatial light modulator. J. Soc. Inf. Disp.
31, 651–656. [Google Scholar]
95).Ueda F.,Kageyama Y.,Iwai D.,Sato K.(2023)焦点表面投影:使用相位-only 空间光调制器扩展投影机景深。J. Soc. Inf. Disp. 31,651–656。[谷歌学术] -
96).Nagase M., Iwai D., Sato K. (2011) Dynamic defocus and occlusion compensation of projected imagery by model-based optimal projector selection in multi-projection environment. Virtual Real.
15, 119–132. [Google Scholar]
96).永濑 M.,岩井 D.,佐藤 K.(2011)多投影环境中基于模型的最优投影仪选择实现的动态失焦和遮挡补偿。虚拟现实. 15,119–132. [谷歌学术] -
97).Tsukamoto J., Iwai D., Kashima K. (2015) Radiometric compensation for cooperative distributed multi-projection system through 2-dof distributed control. IEEE Trans. Vis. Comput. Graph.
21, 1221–1229.
[DOI] [PubMed] [Google Scholar]
97).Tsukamoto J., Iwai D., Kashima K. (2015) 通过 2 自由度分布式控制对合作分布式多投影系统的辐射补偿。IEEE Trans. Vis. Comput. Graph. 21, 1221–1229. [ DOI] [ PubMed] [ Google Scholar] -
98).Uesaka, S. and Amano, T. (2022) Cast-shadow removal for cooperative adaptive appearance manipulation. In ICAT-EGVE 2022 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (eds. Uchiyama, H. and Normand, J.-M.). The Eurographics Association, pp. 11–16. [Google Scholar]
98). Uesaka, S. 和 Amano, T. (2022) 合作自适应外观操纵的铸影去除。在 ICAT-EGVE 2022 - 国际人工智能现实与遥在会议和 Eurographics 虚拟环境研讨会 (Uchiyama, H. 和 Normand, J.-M. 编著)。The Eurographics Association,第 11-16 页。[谷歌学术] -
99).Hiratani K., Iwai D., Kageyama Y., Punpongsanon P., Hiraki T., Sato K. (2023) Shadowless projection mapping using retrotransmissive optics. IEEE Trans. Vis. Comput. Graph.
29, 2280–2290. [DOI] [PubMed] [Google Scholar]
99).Hiratani K.,Iwai D.,Kageyama Y.,Punpongsanon P.,Hiraki T.,Sato K.(2023)使用反透射光学技术的无影投影映射。IEEE Trans. Vis. Comput. Graph. 29,2280-2290。[DOI] [PubMed] [Google Scholar] -
100).Hiratani, K., Iwai, D., Punpongsanon, P. and Sato, K. (2019) Shadowless projector: Suppressing shadows in projection mapping with micro mirror array plate. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 1309–1310. [Google Scholar]
100).Hiratani, K.,Iwai, D.,Punpongsanon, P. 和 Sato, K.(2019)无影投影仪:使用微镜阵列板抑制投影映射中的阴影。在 2019 年 IEEE 虚拟现实与 3D 用户界面会议(VR),第 1309-1310 页。[谷歌学术] -
101).Kiyokawa, M., Okuda, S. and Hashimoto, N. (2019) Stealth projection: Visually removing projectors from dynamic projection mapping. In SIGGRAPH Asia 2019 Posters, SA ’19, Association for Computing Machinery, New York, NY, USA, Article No. 41. [Google Scholar]
101).木川, K., 小川, S. 和 羽織, N. (2019) 隐形投影:从动态投影映射中视觉移除投影仪。在 SIGGRAPH 亚洲 2019 海报,SA’19,美国计算机协会,纽约,纽约,美国,文章编号 41。[谷歌学术] -
102).Kiyokawa, M. and Hashimoto, N. (2021) Dynamic projection mapping with 3D images using volumetric display. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 597–598. [Google Scholar]
102).木川, M. 和橋本, N. (2021) 使用体显示器的 3D 图像动态投影映射。在 2021 年 IEEE 虚拟现实和 3D 用户界面会议摘要和工作坊(VRW)中,第 597-598 页。[谷歌学术] -
103).Watanabe, T. and Hashimoto, N. (2021) Light-field projection for tangible projection mapping. In ACM SIGGRAPH 2021 Posters, SIGGRAPH ’21, Association for Computing Machinery, New York, NY, USA, Article No. 18. [Google Scholar]
103).渡辺 T. 和橋本 N. (2021) 可触投影映射的光场投影。在 ACM SIGGRAPH 2021 海报,SIGGRAPH '21,美国计算机协会,纽约,纽约,美国,文章编号 18。[谷歌学术] -
104).Karitsuka, T. and Sato, K. (2003) A wearable mixed reality with an on-board projector. In The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings, pp. 321–322. [Google Scholar]
104).Karitsuka, T. 和 Sato, K. (2003) 带有车载投影仪的可穿戴混合现实。在第二届 IEEE 和 ACM 国际混合与增强现实研讨会,2003 年。会议论文集,第 321-322 页。[谷歌学术] -
105).Harrison, C., Benko, H. and Wilson, A.D. (2011) Omnitouch: Wearable multitouch interaction everywhere. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, Association for Computing Machinery, New York, NY, USA, pp. 441–450. [Google Scholar]
105).哈里森,C.,本科,H.和威尔逊,A.D.(2011)Omnitouch:无处不在的可穿戴多点触控交互。在 2011 年第 24 届 ACM 用户界面软件和技术年会上发表,UIST '11,美国纽约,纽约州,美国计算机协会,第 441-450 页。[谷歌学术] -
106).Mistry, P., Maes, P. and Chang, L. (2009) Wuw - wear ur world: A wearable gestural interface. In CHI ’09 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’09, Association for Computing Machinery, New York, NY, USA, pp. 4111–4116. [Google Scholar]
106).Mistry, P.,Maes, P.和 Chang, L.(2009)Wuw - wear ur world:一种可穿戴手势界面。在 CHI’09 关于人机交互因素的扩展摘要,CHI EA’09,美国纽约,纽约,美国计算机协会,第 4111-4116 页。[谷歌学术] -
107).Xiao, R., Cao, T., Guo, N., Zhuo, J., Zhang, Y. and Harrison, C. (2018) Lumiwatch: On-arm projected graphics and touch input. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, Association for Computing Machinery, New York, NY, USA, Paper No. 95. [Google Scholar]
107). 小,R.,曹,T.,郭,N.,卓,J.,张,Y. 和 哈里森,C. (2018) Lumiwatch:臂上投影图形和触摸输入。在 2018 年 CHI 计算系统人机因素会议论文集,CHI’18,美国纽约,纽约,纽约,美国,论文编号 95。[谷歌学术] -
108).Cortes G., Marchand E., Brincin G., Lécuyer A. (2018) Mosart: Mobile spatial augmented reality for 3D interaction with tangible objects. Front. Robot. AI
5, 93.
[DOI] [PMC free article] [PubMed] [Google Scholar]
108).Cortes G.,Marchand E.,Brincin G.,Lécuyer A.(2018)Mosart:用于与实体对象进行 3D 交互的移动空间增强现实。Front. Robot. AI 5,93. [DOI] [PMC 免费文章] [PubMed] [Google Scholar] -
109).Wang, C.-H., Yong, S., Chen, H.-Y., Ye, Y.-S. and Chan, L. (2020) Hmd light: Sharing in-VR experience via head-mounted projector for asymmetric interaction. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, UIST ’20, Association for Computing Machinery, New York, NY, USA, pp. 472–486. [Google Scholar]
109).王 C.-H., 永 S., 陈 H.-Y., 叶 Y.-S. 和陈 L. (2020) Hmd 光投影:通过头戴式投影仪实现非对称交互的 VR 内体验。在 33 届年度 ACM 用户界面软件和技术研讨会论文集,UIST '20,美国纽约,纽约州,美国,第 472-486 页。[谷歌学术] -
110).Hartmann, J., Yeh, Y.-T. and Vogel, D. (2020) Aar: Augmenting a wearable augmented reality display with an actuated head-mounted projector. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, UIST ’20, Association for Computing Machinery, New York, NY, USA, pp. 445–458. [Google Scholar]
110).Hartmann, J., Yeh, Y.-T. 和 Vogel, D. (2020) Aar:通过一个可动头戴投影仪增强可穿戴增强现实显示器。在 33 届年度 ACM 用户界面软件和技术研讨会论文集,UIST '20,美国纽约,纽约州,美国,第 445-458 页。[谷歌学术] -
111).Hoffard, J., Miyafuji, S., Pardomuan, J., Sato, T. and Koike, H. (2022) OmniTiles - A User-Customizable Display Using An Omni-Directional Camera Projector System. In ICAT-EGVE 2022 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (eds. Uchiyama, H. and Normand, J.-M.). The Eurographics Association, pp. 149–158. [Google Scholar]
111).霍夫德,J.,宫藤,S.,帕多穆安,J.,佐藤,T.和小池,H.(2022)OmniTiles - 使用全向摄像机投影系统的用户自定义显示器。在 ICAT-EGVE 2022 - 国际人工智能现实与遥在会议和虚拟环境欧几里得研讨会(Uchiyama,H.和 Normand,J.-M. 编著)。欧几里得协会,第 149-158 页。[谷歌学术] -
112).Xu, R., Sato, T., Miyafuji, S. and Koike, H. (2022) Omnilantern: Design and implementations of a portable and coaxial omnidirectional projector-camera system. In Proceedings of the 2022 International Conference on Advanced Visual Interfaces, AVI 2022, Association for Computing Machinery, New York, NY, USA, Article No. 13. [Google Scholar]
112). 徐瑞,佐藤拓,宫藤胜,小池浩(2022)Omnilantern:便携式同轴全向投影相机系统的设计与实现。在 2022 年国际先进视觉界面会议论文集,AVI 2022,美国纽约,纽约州,美国,文章编号 13。[谷歌学术] -
113).Yamamoto K., Iwai D., Tani I., Sato K. (2022) A monocular projector-camera system using modular architecture. IEEE Trans. Vis. Comput. Graph.
29, 5586–5592. [DOI] [PubMed] [Google Scholar]
113).山本耕作,岩井大,谷内一,佐藤健(2022)基于模块化架构的单目投影相机系统。IEEE Trans. Vis. Comput. Graph. 29, 5586–5592. [ DOI] [ PubMed] [ Google Scholar] -
114).Kusakabe Y., Kanazawa M., Nojiri Y., Furuya M., Yoshimura M. (2008) A yc-separation-type projector: High dynamic range with double modulation. J. Soc. Inf. Disp.
16, 383–391. [Google Scholar]
114).草鹿洋一,金泽明,野尻洋,古屋真,吉村真(2008)一种 yc 分离型投影仪:具有双重调制的宽动态范围。信息显示学会杂志 16,383–391。[谷歌学术] -
115).Seetzen H., Heidrich W., Stuerzlinger W., Ward G., Whitehead L., Trentacoste M., et al. (2004) High dynamic range display systems. ACM Trans. Graph.
23, 760–768. [Google Scholar]
115).Seetzen H.,Heidrich W.,Stuerzlinger W.,Ward G.,Whitehead L.,Trentacoste M.,等.(2004)高动态范围显示系统。ACM Trans. Graph. 23,760–768。[谷歌学术] -
116).Kimura, S., Kitamura, M. and Naemura, T. (2007) Emitable: A tabletop surface pervaded with imperceptible metadata. In Second Annual IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP’07), pp. 189–192. [Google Scholar]
116).金丸,S.,北村,M. 和名村,T. (2007) Emitable:一种充满不可察觉元数据的桌面表面。在第二届 IEEE 国际水平交互人机系统研讨会(TABLETOP’07),第 189-192 页。[谷歌学术] -
117).Hiraki T., Fukushima S., Watase H., Naemura T. (2019) Dynamic pvlc: Pixel-level visible light communication projector with interactive update of images and data. ITE Transactions on Media Technology and Applications
7, 160–168. [Google Scholar]
117).Hiraki T.,Fukushima S.,Watase H.,Naemura T.(2019)动态 pvlc:具有图像和数据交互更新的像素级可见光通信投影仪。ITE 媒体技术与应用交易 7,160-168。[谷歌学术] -
118).Hiraki, T., Fukushima, S., Kawahara, Y. and Naemura, T. (2019) Navigatorch: Projection-based robot control interface using high-speed handheld projector. In SIGGRAPH Asia 2019 Emerging Technologies, SA ’19, Association for Computing Machinery, New York, NY, USA, pp. 31–33. [Google Scholar]
118).Hiraki, T.,Fukushima, S.,Kawahara, Y. 和 Naemura, T.(2019)Navigatorch:基于投影的机器人控制界面,使用高速手持投影仪。在 SIGGRAPH 亚洲 2019 新兴技术,SA’19,美国纽约,纽约,美国计算机协会,第 31-33 页。[谷歌学术] -
119).Takefumi Hiraki Y.K., Fukushima S., Naemura T. (2018) Phygital field: An integrated field with physical robots and digital images using projection-based localization and control method. SICE Journal of Control, Measurement, and System Integration
11, 302–311. [Google Scholar]
119).平福武文雄 Y.K.,福島 S.,名村 T.(2018)Phygital field:基于投影定位与控制方法的物理机器人与数字图像集成领域。SICE 控制、测量与系统集成杂志 11,302-311。[谷歌学术] -
120).Miyatake Y., Hiraki T., Iwai D., Sato K. (2023) Haptomapping: Visuo-haptic augmented reality by embedding user-imperceptible tactile display control signals in a projected image. IEEE Trans. Vis. Comput. Graph.
29, 2005–2019.
[DOI] [PubMed] [Google Scholar]
120).宫畑洋一,平井哲也,岩井大辅,佐藤康博(2023)触觉映射:通过在投影图像中嵌入用户难以察觉的触觉显示控制信号实现视觉-触觉增强现实。IEEE Transactions on Visualization and Computer Graphics 29,2005-2019。[DOI] [PubMed] [Google Scholar] -
121).Kumar D., Raut S., Shimasaki K., Senoo T., Ishii I. (2021) Projection-mapping-based object pointing using a high-frame-rate camera-projector system. ROBOMECH J.
8, 8. [Google Scholar]
121).Kumar D.,Raut S.,Shimasaki K.,Senoo T.,Ishii I.(2021)基于投影映射的高帧率相机投影系统进行物体指向。ROBOMECH J. 8,8. [谷歌学术] -
122).Kamei I., Hiraki T., Fukushima S., Naemura T. (2019) Pilc projector: Image projection with pixel-level infrared light communication. IEEE Access
7, 160768–160778. [Google Scholar]
122).上梅一,平木太,福島秀,名村哲(2019)像素级红外光通信图像投影仪:Pilc 投影仪。IEEE Access 7,160768–160778。[谷歌学术] -
123).Bimber, O., Wetzstein, G., Emmerling, A. and Nitschke, C. (2005) Enabling view-dependent stereoscopic projection in real environments. In Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’05), pp. 14–23. [Google Scholar]
123).Bimber, O.,Wetzstein, G.,Emmerling, A.和 Nitschke, C.(2005)在真实环境中实现视点相关立体投影。在第四届 IEEE 和 ACM 国际混合与增强现实研讨会(ISMAR’05),第 14-23 页。[谷歌学术] -
124).Benko, H., Jota, R. and Wilson, A. (2012) Miragetable: Freehand interaction on a projected augmented reality tabletop. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, Association for Computing Machinery, New York, NY, USA, pp. 199–208. [Google Scholar]
124).本科,H.,约塔,R.和威尔逊,A.(2012)Miragetable:在投影增强现实桌面上的自由交互。在 SIGCHI 会议关于人机交互因素的论文集中,CHI '12,美国纽约,纽约,美国计算机协会,第 199-208 页。[谷歌学术] -
125).Kaminokado, T., Iwai, D. and Sato, K. (2019) Augmented environment mapping for appearance editing of glossy surfaces. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 55–65. [Google Scholar]
125).金ノカド,T.,岩井,D. 和 佐藤,K.(2019)用于光泽表面外观编辑的增强环境映射。在 2019 年 IEEE 国际混合与增强现实研讨会(ISMAR),第 55-65 页。[谷歌学术] -
126).Fender, A., Herholz, P., Alexa, M. and Müller, J. (2018) OptiSpace: Automated placement of interactive 3D projection mapping content. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, Association for Computing Machinery, New York, NY, USA, Paper No. 269. [Google Scholar]
126).Fender, A.,Herholz, P.,Alexa, M. 和 Müller, J. (2018) OptiSpace:交互式 3D 投影映射内容的自动放置。在 2018 年 CHI 计算系统人机因素会议论文集,CHI’18,美国纽约,纽约州,美国,论文编号 269。[谷歌学术] -
127).Kimura S., Iwai D., Punpongsanon P., Sato K. (2021) Multifocal stereoscopic projection mapping. IEEE Trans. Vis. Comput. Graph.
27, 4256–4266.
[DOI] [PubMed] [Google Scholar]
127).Kimura S.,Iwai D.,Punpongsanon P.,Sato K.(2021)多焦点立体投影映射。IEEE Trans. Vis. Comput. Graph. 27,4256–4266。[DOI] [PubMed] [Google Scholar] -
128).Ueda T., Iwai D., Hiraki T., Sato K. (2020) Illuminated focus: Vision augmentation using spatial defocusing via focal sweep eyeglasses and high-speed projector. IEEE Trans. Vis. Comput. Graph.
26, 2051–2061.
[DOI] [PubMed] [Google Scholar]
128).Ueda T.,Iwai D.,Hiraki T.,Sato K.(2020)照明焦点:通过焦点扫描眼镜和高速投影仪的空间模糊进行视觉增强。IEEE Trans. Vis. Comput. Graph. 26,2051-2061。[DOI] [PubMed] [Google Scholar] -
129).Miyamoto, J., Koike, H. and Amano, T. (2018) Gaze navigation in the real world by changing visual appearance of objects using projector-camera system. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, VRST ’18, Association for Computing Machinery, New York, NY, USA, pp. 91–98. [Google Scholar]
129).宫本,J.,小池,H. 和天野,T. (2018) 通过投影仪-摄像机系统改变物体视觉外观在现实世界中的视线导航。在 2018 年第 24 届 ACM 虚拟现实软件与技术研讨会论文集,VRST '18,美国纽约,纽约州,美国,第 91-98 页。[谷歌学术] -
130).Ueda T., Iwai D., Sato K. (2021) Illuminatedzoom: spatially varying magnified vision using periodically zooming eyeglasses and a high-speed projector. Opt. Express
29, 16377–16395.
[DOI] [PubMed] [Google Scholar]
130).Ueda T.,Iwai D.,Sato K.(2021)Illuminatedzoom:使用周期性变焦眼镜和高速投影仪实现空间变化放大视觉。Opt. Express 29,16377–16395。[DOI] [PubMed] [Google Scholar] -
131).Hamasaki T., Itoh Y., Hiroi Y., Iwai D., Sugimoto M. (2018) HySAR: Hybrid material rendering by an optical see-through head-mounted display with spatial augmented reality projection. IEEE Trans. Vis. Comput. Graph.
24, 1457–1466.
[DOI] [PubMed] [Google Scholar]
131).Hamasaki T., Itoh Y., Hiroi Y., Iwai D., Sugimoto M. (2018) HySAR:基于光学透视式头戴显示器和空间增强现实投影的混合材料渲染。IEEE Trans. Vis. Comput. Graph. 24, 1457–1466. [DOI] [PubMed] [Google Scholar] -
132).Itoh Y., Kaminokado T., Akşit K. (2021) Beaming displays. IEEE Trans. Vis. Comput. Graph.
27, 2659–2668.
[DOI] [PubMed] [Google Scholar]
132).伊藤洋,神岡貴之,阿克希特·K.(2021)光束显示。IEEE 可视计算与图形学报,27,2659-2668。[DOI] [PubMed] [谷歌学术] -
133).Akşit, K. and Itoh, Y. (2023) Holobeam: Paper-thin near-eye displays. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR), pp. 581–591. [Google Scholar]
133).Akşit, K. 和 Itoh, Y. (2023) Holobeam:超薄近眼显示器。在 2023 年 IEEE 虚拟现实与 3D 用户界面会议(VR),第 581-591 页。[谷歌学术] -
134).Hiroi Y., Watanabe A., Mikawa Y., Itoh Y. (2023) Low-latency beaming display: Implementation of wearable, 133 µs motion-to-photon latency near-eye display. IEEE Trans. Vis. Comput. Graph.
29, 4761–4771.
[DOI] [PubMed] [Google Scholar]
134).Hiroi Y., Watanabe A., Mikawa Y., Itoh Y. (2023) 低延迟波束成形显示器:实现可穿戴的,133 µs 运动到光子近眼显示器。IEEE 计算机视觉与图形学杂志 29,4761–4771. [ DOI] [ PubMed] [ Google Scholar] -
135).Sato, K., Terashima, H., Nishida, S. and Watanabe, Y. (2022) E.S.P.: Extra-sensory puck in air hockey using the projection-based illusion. In SIGGRAPH Asia 2022 Emerging Technologies, SA ’22, Association for Computing Machinery, New York, NY, USA, Article No. 3. [Google Scholar]
135).佐藤 K,寺岛 H,西田 S,渡边 Y. (2022) E.S.P.:利用投影幻觉的空气曲棍球超感官冰球. 在 SIGGRAPH 亚洲 2022 新兴技术,SA’22,美国纽约,纽约,美国计算机协会,文章编号 3. [谷歌学术] -
136).Akiyama R., Yamamoto G., Amano T., Taketomi T., Plopski A., Fujimoto Y., et al. (2020) Illusory light: Perceptual appearance control using a projection-induced illusion. Comput. Graph.
91, 129–140. [Google Scholar]
136). akiyama r.,yamamoto g.,anoto t.,taketomi t.,plopski a.,fujimoto y.,等.(2020)幻觉之光:利用投影诱导的错觉进行感知外观控制。计算机图形学。91,129-140。[谷歌学术] -
137).Nagata, S. and Amano, T. (2019) Gloss enhancement beyond projector performance using the glare illusion. In 26th International Display Workshops, IDW 2019, pp. 974–977. [Google Scholar]
137).永田,S. 和天野,T. (2019) 超越投影仪性能的眩光错觉在词汇增强中的应用。在第 26 届国际显示研讨会,IDW 2019,第 974-977 页。[谷歌学术] -
138).Kawabe T., Fukiage T., Sawayama M., Nishida S. (2016) Deformation lamps: A projection technique to make static objects perceptually dynamic. ACM Trans. Appl. Percept.
13, 10. [Google Scholar]
138).川部 T., 深見 T., 沙山 M., 西田 S. (2016) 变形灯:使静态物体感知动态的投影技术。ACM 应用感知学报 13, 10. [谷歌学术] -
139).Punpongsanon P., Iwai D., Sato K. (2020) Flexeen: Visually manipulating perceived fabric bending stiffness in spatial augmented reality. IEEE Trans. Vis. Comput. Graph.
26, 1433–1439.
[DOI] [PubMed] [Google Scholar]
139).Punpongsanon P.,Iwai D.,Sato K.(2020)Flexeen:在空间增强现实中对感知织物弯曲刚度的视觉操作。IEEE Trans. Vis. Comput. Graph. 26,1433–1439。[DOI] [PubMed] [Google Scholar] -
140).Fukamizu, K., Miyashita, L. and Ishikawa, M. (2020) Elamorph projection: Deformation of 3D shape by dynamic projection mapping. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 164–173. [Google Scholar]
140).福岛水,宫下,石川(2020)Elamorph 投影:通过动态投影映射变形 3D 形状。在 2020 年 IEEE 国际混合与增强现实研讨会(ISMAR),第 164-173 页。[谷歌学术] -
141).Fukiage T., Kawabe T., Nishida S. (2019) Perceptually based adaptive motion retargeting to animate real objects by light projection. IEEE Trans. Vis. Comput. Graph.
25, 2061–2071.
[DOI] [PubMed] [Google Scholar]
141).Fukiage T.,Kawabe T.,Nishida S.(2019)基于感知的适应性运动重定向,通过光线投影使真实物体动画化。IEEE Trans. Vis. Comput. Graph. 25,2061-2071。[DOI] [PubMed] [Google Scholar] -
142).Okutani N., Takezawa T., Iwai D., Sato K. (2018) Stereoscopic capture in projection mapping. IEEE Access
6, 65894–65900. [Google Scholar]
142).奥口谷 N., 武田 T., 伊井 D., 佐藤 K. (2018) 投影映射中的立体捕捉。IEEE Access 6,65894–65900。[谷歌学术] -
143).Kawabe T. (2019) Shadow-based illusion of depth and transparency in printed images. ACM Trans. Appl. Percept.
16, 10. [Google Scholar]
143).川部 T. (2019) 印刷图像中的基于阴影的深度和透明度错觉。ACM 应用感知交易 16,10。[谷歌学术] -
144).Schmidt S., Bruder G., Steinicke F. (2020) Depth perception and manipulation in projection-based spatial augmented reality. Presence
27, 242–256. [Google Scholar]
144).施密特 S.,布鲁德 G.,施泰尼克 F.(2020)基于投影的空间增强现实中的深度感知与操作。存在 27,242–256。[谷歌学术] -
145).Kanamori T., Iwai D., Sato K. (2018) Pseudo-shape sensation by stereoscopic projection mapping. IEEE Access
6, 40649–40655. [Google Scholar]
145).金森敏夫,岩井大辅,佐藤耕平(2018)立体投影映射的伪形状感觉。IEEE Access 6,40649–40655。[谷歌学术] -
146).Sato Y., Hiraki T., Tanabe N., Matsukura H., Iwai D., Sato K. (2020) Modifying texture perception with pseudo-haptic feedback for a projected virtual hand interface. IEEE Access
8, 120473–120488. [Google Scholar]
146).Sato Y.,Hiraki T.,Tanabe N.,Matsukura H.,Iwai D.,Sato K.(2020)通过伪触觉反馈修改投影虚拟手界面的纹理感知。IEEE Access 8,120473-120488。[谷歌学术] -
147).Suzuki Y., Narumi T., Tanikawa T., Hirose M. (2021) Taste in motion: The effect of projection mapping of a boiling effect on food expectation, food perception, and purchasing behavior. Front. Comput. Sci.
3, doi: 10.3389/fcomp.2021.662824. [Google Scholar]
147).铃木勇,成田智,田中毅,广瀬真(2021)动态中的味觉:沸腾效果投影映射对食品预期、食品感知和购买行为的影响。计算机科学前沿 3,doi: 10.3389/fcomp.2021.662824。[谷歌学术] -
148).Fujimoto Y. (2018) Projection mapping for enhancing the perceived deliciousness of food. IEEE Access
6, 59975–59985. [Google Scholar]
148).藤本洋(2018)通过投影映射增强食物感知美味度。IEEE Access 6,59975–59985。[谷歌学术] -
149).Peng, H.-L., Nishida, S. and Watanabe, Y. (2023) Studying user perceptible misalignment in simulated dynamic facial projection mapping. In 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 493–502. [Google Scholar]
149).彭,H.-L.,西田,S. 和 渡边,Y. (2023) 研究模拟动态面部投影映射中的用户可感知偏差。在 2023 年 IEEE 国际混合与增强现实研讨会(ISMAR),第 493-502 页。[谷歌学术] -
150).Nakagawa, S. and Watanabe, Y. (2023) High-frame-rate projection with thousands of frames per second based on the multi-bit superimposition method. In 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 741–750. [Google Scholar]
150).中川,S. 和 渡辺,Y. (2023) 基于多比特叠加法的高帧率投影,每秒数千帧。在 2023 年 IEEE 国际混合与增强现实研讨会(ISMAR),第 741-750 页。[谷歌学术] -
151).Zhong F., Jindal A., Yöntem A.O., Hanji P., Watt S.J., Mantiuk R.K. (2021) Reproducing reality with a high-dynamic-range multi-focal stereo display. ACM Trans. Graph.
40, 241. [Google Scholar]
151). 中福,金达尔,尹泰姆,韩吉,沃特,曼图克(2021)使用高动态范围多焦点立体显示重现现实。ACM 图形传输。40,241。[谷歌学术] -
152).Morimoto T., Numata A., Fukuda K., Uchikawa K. (2021) Luminosity thresholds of colored surfaces are determined by their upper-limit luminances empirically internalized in the visual system. J. Vis., 21, 3. [DOI] [PMC free article] [PubMed] [Google Scholar]
152).Morimoto T.,Numata A.,Fukuda K.,Uchikawa K.(2021)彩色表面的亮度阈值由视觉系统中经验内化的上限亮度决定。J. Vis.,21,3。[DOI] [PMC 免费文章] [PubMed] [Google Scholar] -
153).Amano, T. and Kubo, R. (2022) Reproduction of multiple mirror-based arbitrary lighting environment. In 2022 EuroXR Conference, pp. 81–85. [Google Scholar]
153).阿马诺,T. 和久保,R. (2022) 基于多镜任意照明环境的再现。在 2022 年 EuroXR 会议,第 81-85 页。[谷歌学术] -
154).Iwai, D. (2021) Is projection mapping natural? towards physical world augmentation consistent with light field context. In 28th International Display Workshops, IDW 2021, pp. 964–967. [Google Scholar]
154).岩井,D.(2021)投影映射是自然的吗?与光场上下文一致的物理世界增强。在第 28 届国际显示研讨会,IDW 2021,第 964-967 页。[谷歌学术] -
155).Kubo, R. and Amano, T. (2020) A study on imperceptible projection for seamless appearance manipulation. In 27th International Display Workshops, IDW 2020, pp. 659–662. [Google Scholar]
155).Kubo, R. 和 Amano, T. (2020) 关于无缝外观操控不可感知投影的研究。在第 27 届国际显示研讨会,IDW 2020,第 659-662 页。[谷歌学术] -
156).Takeuchi, M., Iwai, D. and Sato, K. (2023) Projection mapping in the light: A preliminary attempt to substitute projectors for room lights. In 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 653–654. [Google Scholar]
156).武内,M.,岩井,D.,佐藤,K.(2023)光中的投影映射:用投影仪替代室内照明的初步尝试。在 2023 年 IEEE 虚拟现实与 3D 用户界面会议摘要和工作坊(VRW),第 653-654 页。[谷歌学术] -
157).Iwai D., Aoki M., Sato K. (2019) Non-contact thermo-visual augmentation by ir-rgb projection. IEEE Trans. Vis. Comput. Graph.
25, 1707–1716.
[DOI] [PubMed] [Google Scholar]
157).Iwai D.,Aoki M.,Sato K.(2019)红外 RGB 投影的非接触式热视觉增强。IEEE Trans. Vis. Comput. Graph. 25,1707-1716。[DOI] [PubMed] [Google Scholar] -
158).Gao, L., Irani, P., Subramanian, S., Prabhakar, G., Martinez Plasencia, D. and Hirayama, R. (2023) DataLev: Mid-air data physicalisation using acoustic levitation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23, Association for Computing Machinery, New York, NY, USA, Article No. 312. [Google Scholar]
158).高,L.,伊拉尼,P.,苏布拉马尼亚,S.,普拉哈卡,G.,马丁内斯·普拉森西亚,D. 和平山,R.(2023)DataLev:利用声悬浮进行空中数据物理化。在 2023 年 CHI 计算系统人因工程会议论文集,CHI '23,美国纽约,纽约州,美国,文章编号 312。[谷歌学术] -
159).Hasegawa K., Shinoda H. (2018) Aerial vibrotactile display based on multiunit ultrasound phased array. IEEE Trans. Haptics
11, 367–377. [DOI] [PubMed] [Google Scholar]
159).Hasegawa K.,Shinoda H.(2018)基于多单元超声波相控阵的空中振动触觉显示。IEEE 触觉学报 11,367-377。[DOI] [PubMed] [Google Scholar] -
160).Shinogi, R., Matsukura, H. and Ishida, H. (2019) Smelling screen: Application to a museum exhibition and a challenge for scaling up. In 2019 IEEE International Symposium on Olfaction and Electronic Nose (ISOEN), doi: 10.1109/ISOEN.2019.8823304. [Google Scholar]
160).Shinogi, R.,Matsukura, H. 和 Ishida, H.(2019)气味屏幕:应用于博物馆展览和扩大规模的挑战。在 2019 年 IEEE 国际嗅觉与电子鼻研讨会(ISOEN),doi:10.1109/ISOEN.2019.8823304。[谷歌学术] -
161).Yanagida, Y., Nakano, T. and Watanabe, K. (2019) Towards precise spatio-temporal control of scents and air for olfactory augmented reality. In 2019 IEEE International Symposium on Olfaction and Electronic Nose (ISOEN), doi: 10.1109/ISOEN.2019.8823180. [Google Scholar]
161). Yanagida, Y.,Nakano, T. 和 Watanabe, K. (2019) 向精确控制气味和空气的嗅觉增强现实迈进。在 2019 年 IEEE 国际嗅觉与电子鼻研讨会(ISOEN),doi: 10.1109/ISOEN.2019.8823180。[谷歌学术]