kaolin.graphics

class DifferentiableRenderer[source]

Base class for differentiable renderers. All major components of the graphics processing pipeline have been instantiated as abstract methods.

A differentiable renderer takes in vertex geometry, faces (usually triangles), and optionally texture (and any other information deemed relevant), and implements the following main steps.

  • Lighting (ambient and optionally, directional lighting, i.e., diffuse/Lambertian reflection, and optionally specular reflection)

  • (Vertex) Shading (Gouraud/Phong/etc.)

  • Geometric transformation (usually, this is needed to view the object to be rendered from a specific viewing angle, etc.) (the geometric transformations are usually applied AFTER lighting, as it is easier to perform lighting in the object frame, rather than at the scene level)

  • Projection (perspective/orthographic/weak-orthographic(affine)) from 3D (camera) coordinates to 2D (image) coordinates.

  • Rasterization to convert the primitives (usually triangles) to pixels, and determine texture (when available).

class NeuralMeshRenderer(image_size: int = 256, anti_aliasing: bool = True, bg_color: torch.Tensor = tensor([0., 0., 0.]), fill_back: bool = True, camera_mode: str = 'projection', K=None, rmat=None, tvec=None, perspective_distort: bool = True, viewing_angle: float = 30.0, camera_direction: torch.Tensor = tensor([0., 0., 1.]), near: float = 0.1, far: float = 100, light_intensity_ambient: float = 0.5, light_intensity_directional: float = 0.5, light_color_ambient: torch.Tensor = tensor([1., 1., 1.]), light_color_directional: torch.Tensor = tensor([1., 1., 1.]), light_direction: torch.Tensor = tensor([0., 1., 0.]), device: str = 'cpu')[source]

A class implementing the emph{Neural Mesh Renderer} from the following CVPR 2018 paper:

Neural 3D Mesh Renderer Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada Link: https://arxiv.org/abs/1711.07566

class SoftRenderer(image_size: int = 256, anti_aliasing: bool = True, bg_color: torch.Tensor = tensor([0., 0., 0.]), fill_back: bool = True, camera_mode: str = 'projection', K=None, rmat=None, tvec=None, perspective_distort: bool = True, sigma_val: float = 1e-05, dist_func: str = 'euclidean', dist_eps: float = 0.0001, gamma_val: float = 0.0001, aggr_func_rgb: str = 'softmax', aggr_func_alpha: str = 'prod', texture_type: str = 'surface', viewing_angle: float = 30.0, viewing_scale: float = 1.0, eye: torch.Tensor = None, camera_direction: torch.Tensor = tensor([0., 0., 1.]), near: float = 0.1, far: float = 100, light_mode: str = 'surface', light_intensity_ambient: float = 0.5, light_intensity_directional: float = 0.5, light_color_ambient: torch.Tensor = tensor([1., 1., 1.]), light_color_directional: torch.Tensor = tensor([1., 1., 1.]), light_direction: torch.Tensor = tensor([0., 1., 0.]), device: str = 'cpu')[source]

A class implementing the emph{Soft Renderer} from the following ICCV 2019 paper:

Soft Rasterizer: A differentiable renderer for image-based 3D reasoning Shichen Liu, Tianye Li, Weikai Chen, and Hao Li Link: https://arxiv.org/abs/1904.01786

class DIBRenderer[source]

Placeholder class for DIB-Renderer implementation.

A differentiable renderer takes in vertex geometry, faces (usually triangles), and optionally texture (and any other information deemed relevant), and implements the following main steps.

  • Lighting (ambient and optionally, directional lighting, i.e., diffuse/Lambertian reflection, and optionally specular reflection)

  • (Vertex) Shading (Gouraud/Phong/etc.)

  • Geometric transformation (usually, this is needed to view the object to be rendered from a specific viewing angle, etc.) (the geometric transformations are usually applied AFTER lighting, as it is easier to perform lighting in the object frame, rather than at the scene level)

  • Projection (perspective/orthographic/weak-orthographic(affine)) from 3D (camera) coordinates to 2D (image) coordinates.

  • Rasterization to convert the primitives (usually triangles) to pixels, and determine texture (when available).

apply_ambient_light(face_vertices: torch.Tensor, textures: torch.Tensor, ambient_intensity: float = 1.0, ambient_color: torch.Tensor = tensor([1., 1., 1.]))[source]

Computes and applies ambient lighting to a mesh, given faces and face textures.

Parameters

  • face_vertices (torch.Tensor) – A tensor containing a list of (per-face) vertices of the mesh (shape: B \(\times\) num_faces \(\times 9\)). Here, \(B\) is the batchsize, num_faces is the number of faces in the mesh, and since each face is assumed to be a triangle, it has 3 vertices, and hence 9 coordinates in total.

  • textures (torch.Tensor) – TODO: Add docstring

  • ambient_intensity (float) – Intensity of ambient light (in the range \(\left[0, 1\right]\)). If the values provided are outside this range, we clip them so that they fall in range.

  • ambient_color (torch.Tensor) – Color of the ambient light (R, G, B) (shape: \(3\))

Returns

Updated textures, with ambient lighting

applied (shape: same as input textures) #TODO: Update docstring

Return type

textures (torch.Tensor)

apply_directional_light(face_vertices: torch.Tensor, textures: torch.Tensor, directional_intensity: float = 1.0, directional_color: torch.Tensor = tensor([1., 1., 1.]), direction: torch.Tensor = tensor([0., 1., 0.]))[source]

Computes and applies directional lighting to a mesh, given faces and face textures.

Parameters

  • face_vertices (torch.Tensor) – A tensor containing a list of (per-face) vertices of the mesh (shape: B \(\times\) num_faces \(\times 9\)). Here, \(B\) is the batchsize, num_faces is the number of faces in the mesh, and since each face is assumed to be a triangle, it has 3 vertices, and hence 9 coordinates in total.

  • textures (torch.Tensor) – TODO: Add docstring

  • directional_intensity (float) – Intensity of directional light (in the range \(\left[0, 1\right]\)). If the values provided are outside this range, we clip them so that they fall in range.

  • directional_color (torch.Tensor) – Color of the directional light (R, G, B) (shape: \(3\)).

  • direction (torch.Tensor) – Direction of light from the light source. (default: \(\left( 0, 1, 0 \right)^T\))

Returns

A light tensor, which can be elementwise

multiplied with the textures, to obtain the mesh with lighting applied (shape: B \(\times\) num_faces \(\times 1 \times 1 \times 1 \times 3\))

Return type

light (torch.Tensor)

get_eye_from_spherical_coords(distance: torch.Tensor, elevation: torch.Tensor, azimuth: torch.Tensor, degrees: Optional[bool] = True)[source]

Returns the Cartesian position of the eye of the camera, given spherical coordinates (distance, azimuth, and elevation).

Parameters

  • distance (torch.Tensor) – Distance of the eye from the object.

  • elevation (torch.Tensor) – Elevation angle.

  • azimuth (torch.Tensor) – Azimuth angle.

  • degrees (bool, optional) – Bool to indicate that azimuth and elevation are specified in degrees (default: True).

Returns

Position of the “eye” of the camera.

Return type

(torch.Tensor)