Introduction

Robot-assisted minimally invasive surgery (RMIS) has received an increasing amount of attention because of its unique advantages compared with traditional open surgery.[1-5] In RMIS, laparoscopes used to display the surgical scenario on a screen are usually held by a robotic arm instead of an assistant. Surgeons need to frequently pause the operation of surgical instruments and adjust the laparoscope to provide a better field of view (FOV). This distracts the surgeon during the surgery, thereby prolonging the operation time and causing surgeon fatigue.
A laparoscopic control method to adjust the FOV of laparoscopy by tracking surgical instruments automatically during the surgical procedure needs to be developed. Currently, rigid laparoscopy is widely applied to RMIS with automatic FOV adjusting algorithms. Yang et al. proposed a region-based visual servoing method to automatically manipulate laparoscopy with colored markers, which can improve the control efficiency and safety of FOV.[6] An autonomous surgical instrument tracking method without any markers was further proposed based on the visual tracking space vector.[7] However, these methods encounter the workspace problem. When operating a rigid laparoscopy using a robotic arm, avoiding collision between the robot arm and the other surgical instruments is necessary. Therefore, using a robotic arm to operate a rigid laparoscope in a narrow workspace is difficult, leading to the limited adjustment workspace of rigid laparoscopy.
Continuum manipulators have been widely applied in robotic surgical applications due to their higher dexterity and less workspace required.[8-10] Recently, continuum manipulators have been used for automatic FOV adjustment with visual servoing in RMIS.[11] However, the dynamics of continuum manipulator are always highly nonlinear and high-dimensional due to the mechanical compliance of its structure.[12] These characteristics bring challenges to the precise control of continuum manipulators. Existing methods usually simplify the continuum manipulator based on physical assumptions in establishing dynamic models, such as piecewise constant curvature model, pseudo-rigid-body, quasistatic and simplified geometry. [13-20] Assumptions in these simplified models may lead to deviation under actual conditions and inaccurate results, which are not feasible for use in practice, especially for scenarios with high precision requirements. 
Recently, data-driven control methods such as neural networks and reinforcement learning, have shown great potential for controlling continuum manipulators.[21,22] The advantage of these methods lies in the input-output mapping of the system derived from sensing data without analytical modelling and complex computation. Given enough collected input-output data, data-driven models can describe the behavior of the system over its entire operating range. However, these methods usually require many tuning parameters and repeated trials to establish accurate models. Other concerns include low real-time performance and computational complexity.[23] Koopman operator provides an alternative solution for establishing the dynamic model of a continuum manipulator based on its unique linear structure.[24,25] Koopman operator lifts the nonlinear dynamic model of the system into an infinite-dimensional space and evolves the state functions, which are also called observation functions in the new space. In this way, the dynamic model of the nonlinear system can be easily propagated in a linear manner, relying on input-output data only. As a result, linear control methods can be applied to control the continuum manipulator with high precision.
Apart from accurate system identification, a close-loop control with high precision visual feedback is also important. Visual feedback in laparoscopic instruments tracking can be divided into two types: marked methods and unmarked methods. Marked methods manually add a characteristic marker on the instrument for easy detection. Although this method can localize the target quickly, uncertainty exits due to the presence of blood and gas during surgery. Furthermore, this method provides surgeons with a poor experience and has a low tracking precision because markers are usually located at a non-client rod of the instrument. Unmarked methods usually choose the whole metal part of the instrument as the detecting area and then detect the area as an object detection task using a deep learning algorithm.[26] However, this method requires surgeons to focus on different points of operation at different stages of surgery, and the method is often not flexible enough. For example, the ultrasonic knife in surgery used to resect tissue and the focused point should be the tip of the instrument. Scissors are used to clamp tissues or needles, so the focused point should be the center of the clasper. Therefore, the unmarked methods result in a less accurate visual feedback.
In this present work, we focus on autonomous control of a continuum laparoscope to adjust the FOV and keep the surgical instruments at the view center in RMIS. To address this critical issue, we proposed an automatic surgical instrument tracking framework based on a Koopman-based control scheme and learning-based vision feedback. This framework can be divided into two units. The first unit is the data-driven system identification unit, which applies the Koopman operator to transfer a nonlinear dynamics system into a linear closed-loop control. Unlike the Taylor-based method, we introduce the Chebyshev polynomials to choose observation functions.[27,28] Chebyshev polynomials is a global approximation method dependent on high-order derivatives of the system state as existing methods. The approximation error of the proposed method is also analyzed. A linear quadratic regulator (LQR) controller is further used for real-time control based on the linear representation of the continuum laparoscope.