Parameterizing shapes with Anny¶

Instantiate the model¶

Anny is shipped as a Python package that can be easily installed (see the README for details).

In [1]:
from IPython.display import Markdown, display
import torch
import roma # A PyTorch library useful to deal with space transformations.
import anny # The main library for the Anny model.
import trimesh # For 3D mesh visualization.

# Instantiate the model, with all shape parameters available.
# Remark: the first instantiation may take a while. Latter calls will be faster thanks to caching.
anny_model = anny.create_fullbody_model(eyes=True, tongue=False, all_phenotypes=True, local_changes=True, remove_unattached_vertices=True)
# Use 32bit floating point precision on the CPU for this demo.
dtype = torch.float32
device = torch.device('cpu')
anny_model = anny_model.to(device=device, dtype=dtype)

# A simple transform to get a better view angle in 3D mesh visualizations.
trimesh_scene_transform = roma.Rigid(linear=roma.euler_to_rotmat('x', [-90.], degrees=True), translation=None).to_homogeneous().cpu().numpy()

Warp 1.8.1 initialized:
   CUDA Toolkit 12.8, Driver 13.0
   Devices:
     "cpu"      : "x86_64"
     "cuda:0"   : "NVIDIA RTX 1000 Ada Generation Laptop GPU" (6 GiB, sm_89, mempool enabled)
   Kernel cache:
some_path

Template mesh¶

This is the template mesh of Anny:

In [2]:
display(Markdown(f"{anny_model.template_vertices.shape[0]} vertices -- {anny_model.faces.shape[0]} faces composed of {anny_model.faces.shape[1]} vertices each."))
trimesh.Trimesh(vertices=anny_model.template_vertices.cpu().numpy(), faces=anny_model.faces.cpu().numpy()).apply_transform(trimesh_scene_transform).show()

13492 vertices -- 13486 faces composed of 4 vertices each.

Out[2]:

Shape parameterization¶

Anny can model a diversity of morphologies. We follow MakeHuman terminology, and parameterize diversity of body shapes using a set of phenotype parameters, typically between 0 and 1.

Note: the values african, caucasian and asian parameters are normalized so that they sum to 1.

Word of caution regarding phenotypes: Phenotypes are based on preconceptions of artists regarding particular human traits. As a result, they encode by design stereotypes of MakeHuman artists, and one should not expect phenotype parameters to faithfully encode identity-related characteristics, such as gender, age or ethnicity.

In [3]:
# List phenotype parameters
Markdown("**List of phenotype parameters**: " + ", ".join([f"{label}" for label in anny_model.phenotype_labels]))
Out[3]:

List of phenotype parameters: gender, age, muscle, weight, height, proportions, cupsize, firmness, african, asian, caucasian

Example¶

Here we show an example of how the age parameter influences the resulting mesh.

In [4]:
batch_size = 5  # We can process multiple bodies at once in a batch. 

phenotype_kwargs = {key : torch.full((batch_size,), fill_value=0.5, dtype=dtype, device=device) for key in anny_model.phenotype_labels}
phenotype_kwargs['age'] = torch.linspace(0., 1., batch_size, dtype=dtype, device=device) # Example: vary the age parameter across the batch.
output = anny_model(phenotype_kwargs=phenotype_kwargs)

scene = trimesh.Scene()
for i in range(batch_size):
    # Create a mesh for each body in the batch.
    mesh = trimesh.Trimesh(vertices=output['vertices'][i].squeeze().cpu().numpy(), faces=anny_model.faces.cpu().numpy())
    transform = roma.Rigid(linear=None, translation=torch.tensor([i * 1., 0., 0.], dtype=dtype, device=device)).to_homogeneous().cpu().numpy()
    scene.add_geometry(mesh, transform=transform)

scene.apply_transform(trimesh_scene_transform)  # Rotate the scene to have a better view.
scene.show()  # This will open a window to visualize the scene with all the bodies in it.

Module anny.skinning.warp_skinning 4408221 load on device 'cpu' took 3.16 ms  (cached)
Out[4]:

Local changes¶

Additionnally one specify some more local morphological changes. Local change parameters values are typically expected to be chosen between -1 and 1, but one can use values outside this range to extrapolate changes even further.

Note: it is easy to produce unrealistic meshes when using significant local changes.

In [5]:
display(Markdown("**List of local changes parameters:** " + ", ".join(anny_model.local_change_labels)))

List of local changes parameters: l-lowerarm-scale-depth-incr, r-lowerarm-scale-depth-incr, l-upperarm-scale-horiz-incr, r-upperarm-scale-horiz-incr, l-upperarm-fat-incr, r-upperarm-fat-incr, measure-upperarm-length-incr, l-lowerarm-fat-incr, r-lowerarm-fat-incr, l-lowerarm-scale-horiz-incr, r-lowerarm-scale-horiz-incr, l-lowerarm-muscle-incr, r-lowerarm-muscle-incr, l-upperarm-shoulder-muscle-incr, r-upperarm-shoulder-muscle-incr, measure-upperarm-circ-incr, l-upperarm-scale-depth-incr, r-upperarm-scale-depth-incr, l-lowerarm-scale-vert-incr, r-lowerarm-scale-vert-incr, measure-lowerarm-length-incr, l-upperarm-muscle-incr, r-upperarm-muscle-incr, l-upperarm-scale-vert-incr, r-upperarm-scale-vert-incr, breast-point-incr, nipple-point-incr, nipple-size-incr, breast-dist-incr, breast-volume-vert-up, breast-trans-up, buttocks-volume-incr, l-cheek-trans-up, r-cheek-trans-up, l-cheek-volume-incr, r-cheek-volume-incr, l-cheek-inner-incr, r-cheek-inner-incr, l-cheek-bones-incr, r-cheek-bones-incr, chin-jaw-drop-incr, chin-height-incr, chin-prominent-incr, chin-bones-incr, chin-width-incr, chin-cleft-incr, chin-prognathism-incr, l-ear-lobe-incr, r-ear-lobe-incr, l-ear-shape-round, r-ear-shape-round, l-ear-trans-forward, r-ear-trans-forward, l-ear-rot-forward, r-ear-rot-forward, l-ear-trans-up, r-ear-trans-up, l-ear-scale-vert-incr, r-ear-scale-vert-incr, l-ear-flap-incr, r-ear-flap-incr, l-ear-shape-triangle, r-ear-shape-triangle, l-ear-scale-incr, r-ear-scale-incr, l-ear-scale-depth-incr, r-ear-scale-depth-incr, l-ear-wing-incr, r-ear-wing-incr, eyebrows-angle-up, eyebrows-trans-up, eyebrows-trans-forward, l-eye-corner1-up, r-eye-corner1-up, l-eye-height1-incr, r-eye-height1-incr, l-eye-scale-incr, r-eye-scale-incr, l-eye-push1-out, r-eye-push1-out, l-eye-eyefold-up, r-eye-eyefold-up, l-eye-height3-incr, r-eye-height3-incr, l-eye-height2-incr, r-eye-height2-incr, l-eye-bag-incr, r-eye-bag-incr, l-eye-push2-out, r-eye-push2-out, l-eye-trans-up, r-eye-trans-up, l-eye-bag-height-incr, r-eye-bag-height-incr, l-eye-epicanthus-out, r-eye-epicanthus-out, l-eye-eyefold-angle-up, r-eye-eyefold-angle-up, l-eye-bag-out, r-eye-bag-out, l-eye-corner2-up, r-eye-corner2-up, l-eye-trans-out, r-eye-trans-out, l-foot-scale-horiz-incr, r-foot-scale-horiz-incr, l-foot-trans-out, r-foot-trans-out, l-foot-scale-vert-incr, r-foot-scale-vert-incr, l-foot-trans-forward, r-foot-trans-forward, l-foot-scale-depth-incr, r-foot-scale-depth-incr, l-foot-scale-incr, r-foot-scale-incr, l-foot-trans-up, r-foot-trans-up, measure-ankle-circ-incr, forehead-temple-incr, forehead-trans-forward, forehead-scale-vert-incr, forehead-nubian-incr, l-hand-trans-out, r-hand-trans-out, l-hand-fingers-length-incr, r-hand-fingers-length-incr, l-hand-scale-incr, r-hand-scale-incr, l-hand-fingers-diameter-incr, r-hand-fingers-diameter-incr, l-hand-fingers-distance-incr, r-hand-fingers-distance-incr, measure-wrist-circ-incr, head-angle-out, head-scale-vert-incr, head-trans-forward, head-trans-up, head-fat-incr, head-trans-out, head-back-scale-depth-incr, head-age-incr, head-scale-depth-incr, head-scale-horiz-incr, hip-trans-out, hip-scale-horiz-incr, hip-scale-depth-incr, hip-waist-up, hip-trans-forward, hip-scale-vert-incr, hip-trans-up, measure-calf-circ-incr, measure-lowerleg-height-incr, l-upperleg-fat-incr, r-upperleg-fat-incr, l-lowerleg-muscle-incr, r-lowerleg-muscle-incr, l-leg-valgus-incr, r-leg-valgus-incr, l-lowerleg-scale-depth-incr, r-lowerleg-scale-depth-incr, l-lowerleg-scale-vert-incr, r-lowerleg-scale-vert-incr, l-upperleg-scale-depth-incr, r-upperleg-scale-depth-incr, l-upperleg-scale-horiz-incr, r-upperleg-scale-horiz-incr, upperlegs-height-incr, l-lowerleg-fat-incr, r-lowerleg-fat-incr, l-lowerleg-scale-horiz-incr, r-lowerleg-scale-horiz-incr, lowerlegs-height-incr, measure-upperleg-height-incr, l-upperleg-muscle-incr, r-upperleg-muscle-incr, measure-knee-circ-incr, measure-thigh-circ-incr, l-upperleg-scale-vert-incr, r-upperleg-scale-vert-incr, mouth-lowerlip-height-incr, mouth-scale-horiz-incr, mouth-cupidsbow-incr, mouth-laugh-lines-out, mouth-scale-depth-incr, mouth-lowerlip-volume-incr, mouth-angles-up, mouth-upperlip-ext-up, mouth-lowerlip-width-incr, mouth-trans-forward, mouth-upperlip-width-incr, mouth-scale-vert-incr, mouth-dimples-out, mouth-upperlip-height-incr, mouth-upperlip-volume-incr, mouth-cupidsbow-width-incr, mouth-lowerlip-ext-up, mouth-upperlip-middle-up, mouth-trans-up, mouth-trans-out, mouth-lowerlip-middle-up, mouth-philtrum-volume-incr, measure-neck-circ-incr, measure-neck-height-incr, neck-scale-depth-incr, neck-trans-forward, neck-double-incr, neck-trans-out, neck-back-scale-depth-incr, neck-scale-horiz-incr, neck-scale-vert-incr, neck-trans-up, nose-nostrils-angle-up, nose-flaring-incr, nose-curve-convex, nose-scale-depth-incr, nose-scale-vert-incr, nose-point-width-incr, nose-scale-horiz-incr, nose-width1-incr, nose-trans-forward, nose-trans-up, nose-hump-incr, nose-nostrils-width-incr, nose-trans-out, nose-greek-incr, nose-point-up, nose-width2-incr, nose-width3-incr, nose-septumangle-incr, nose-base-up, nose-compression-uncompress, nose-volume-incr, bulge-incr, pelvis-tone-incr, stomach-pregnant-incr, stomach-navel-out, stomach-navel-up, stomach-tone-incr, torso-trans-forward, torso-muscle-pectoral-incr, measure-underbust-circ-incr, torso-scale-horiz-incr, torso-scale-depth-incr, measure-napetowaist-dist-incr, measure-waisttohip-dist-incr, measure-shoulder-dist-incr, measure-hips-circ-incr, torso-vshape-incr, torso-trans-up, torso-muscle-dorsi-incr, measure-waist-circ-incr, measure-frontchest-dist-incr, torso-scale-vert-incr, measure-bust-circ-incr, torso-trans-out

Local change example¶

We show here the effect of the stomach-pregnant-incr local change parameter, as an example.

In [6]:
batch_size = 3  # We can process multiple faces at once in a batch.
pose_parameters = roma.Rigid.identity(dim=3, batch_shape=(batch_size, anny_model.bone_count)).to_homogeneous()
phenotype_kwargs = {key : torch.full((batch_size,), fill_value=0.5) for key in anny_model.phenotype_labels}
# In this example, we start from a stereotypical young adult woman mesh
phenotype_kwargs['age'].fill_(0.67) 
phenotype_kwargs['gender'].fill_(1.)

local_changes = {'stomach-pregnant-incr': torch.linspace(0, 1., batch_size)}  # Example: vary the upperarm fat increment across the batch.
output = anny_model(phenotype_kwargs=phenotype_kwargs, local_changes_kwargs=local_changes)

scene = trimesh.Scene()
for i in range(batch_size):
    # Create a mesh for each face in the batch.
    mesh = trimesh.Trimesh(vertices=output['vertices'][i].squeeze().cpu().numpy(), faces=anny_model.faces.cpu().numpy())
    transform = roma.Rigid(linear=None, translation=torch.tensor([i * 1., 0., 0.])).to_homogeneous().cpu().numpy()
    scene.add_geometry(mesh, transform=transform)
scene.apply_transform(trimesh_scene_transform)  # Rotate the scene to have a better view.
scene.show()  # This will open a window to visualize the scene with all the faces in
Out[6]:

Phenotype distribution¶

In [7]:
import anny.shape_distribution
import anny.anthropometry
import matplotlib.pyplot as plt

phenotype_distribution = anny.shape_distribution.SimpleShapeDistribution(anny_model,
            morphological_age_distribution=torch.distributions.Uniform(low=0.0, high=60.0))

real_age, phenotype_kwargs = phenotype_distribution.sample(batch_size=200)
output = anny_model(phenotype_kwargs=phenotype_kwargs)

scene = trimesh.Scene()
i = -1
for u in range(4):
    for v in range(5):
        i += 1
        assert i < output['vertices'].shape[0], "Batch size is too small for the grid."
        # Create a mesh for each face in the batch.
        mesh = trimesh.Trimesh(vertices=output['vertices'][i].squeeze().cpu().numpy(), faces=anny_model.faces.cpu().numpy())
        transform = roma.Rigid(linear=None, translation=torch.tensor([v * 1., u * 1., 0.], dtype=dtype, device=device)).to_homogeneous().cpu().numpy()
        scene.add_geometry(mesh, transform=transform)
scene.apply_transform(trimesh_scene_transform)  # Rotate the scene to have a better view.
scene.show()  # This will open a window to visualize the scene with all the faces in
Out[7]:

Body measures¶

We additionnally provide a class to estimate some anthropometric measurements, assuming a body buoyancy of .98 in water.

In [8]:
real_age, phenotype_kwargs = phenotype_distribution.sample(batch_size=1000)
output = anny_model(phenotype_kwargs=phenotype_kwargs)

measurements = anny.anthropometry.Anthropometry(anny_model)
measures = measurements(output['rest_vertices'])

fig, axes = plt.subplots(1,3, squeeze=True, figsize=(10, 5))
axes[0].scatter(real_age.cpu().numpy(), measures['height'].cpu().numpy())
axes[0].set_xlabel('Morphological age (year)')
axes[0].set_ylabel('Height (m)')
axes[1].scatter(real_age.cpu().numpy(), measures['waist_circumference'].cpu().numpy())
axes[1].set_xlabel('Morphological age (year)')
axes[1].set_ylabel('Waist circumference (m)')
axes[2].scatter(real_age.cpu().numpy(), measures['bmi'].cpu().numpy())
axes[2].set_xlabel('Morphological age (year)')
axes[2].set_ylabel('Body Mass Index estimate')
fig.tight_layout()
plt.show()
No description has been provided for this image