I would like to inquire as to whether the my logic for finding a 3D point in space with reference to the camera depth optical frame is correct. The location of the point in space is retrieved from a pixel location in the depth image (depth_image[row][column]). The ROS depth image topic is image_rect_raw, so to my understanding this is a rectified BUT distorted image, therefore the parameters if the camera property (camera intrinsics e.t.c) must be used to attain the undistorted 3D point.

Here are relevant snippets of code. The conversion of ROS depth image data to openCV depth image:

"""convert to opencv depth image"""
                    depth_image = self.bridge.imgmsg_to_cv2(self.depth_img, desired_encoding='16UC1')

The callback for Camera info messages so as to get K and D matrix to undistort rectified depth image:

   """subscriber to Camerainfo messages"""
    def callback0(self,msg):
        self.camera_intrinsics = msg

        """Reshape K matrix"""
        self.camera_matrix = np.array(self.camera_intrinsics.K).reshape(3, 3)

The snippet of code which finds a point in 3D space with reference to the depth camera optical frame using the z_depth value of a pixel location in the depth image and the K and D matrixes:

 """Get Z_depth in meters of pixel in depth image"""
    z_depth = depth_image[row][col]/1000

"""Obtain the distortion coefficients from the distortion matrix/D matrix:"""
   distortion_coeffs = np.array(self.camera_intrinsics.D)

 """Apply distortion correction to the image coordinates (row, column) using the distortion coefficients.
  This step will remove the distortion effects and provide undistorted image coordinates:"""
 undistorted_coords = cv2.undistortPoints(np.array([(col, row)], dtype=np.float32),self.camera_matrix, distortion_coeffs)
 undistorted_x = undistorted_coords[0, 0, 0]
 undistorted_y = undistorted_coords[0, 0, 1]

"""Convert the undistorted image coordinates to normalized device coordinates (NDC) by 
subtracting the principal point and dividing by the focal length"""
 x_ndc = (undistorted_x - self.camera_intrinsics.K[2]) / self.camera_intrinsics.K[0]
  y_ndc = (undistorted_y - self.camera_intrinsics.K[5]) / self.camera_intrinsics.K[4]

 """Scale the normalized device coordinates to obtain the 3D coordinates in camera coordinate space"""
   x_cam = x_ndc * (z_depth) / 1.0
   y_cam = y_ndc * (z_depth) / 1.0

print("CLASS_NAME: ", str(self.class_names[class_index]), " ,3D POINT in depth_camera_optical_frame: ",[x_cam ,y_cam,z_depth])

I will provide any other information necessary to help this kind community provide me with help.