Character Generation Pipeline

Import Mixamo Characters as Virtual Community Avatars

Step 1: Download FBX model from Mixamo

Click download, select FBX Binary (.fbx) and T-pose.

Step 2: Process FBX model with Blender

1. Import FBX into Blender
   File -> Import -> FBX (.fbx), select one file at a time.

2. Merge meshes if needed
   Select all meshes in the outliner, right-click in the 3D Viewport, then click Join.
   (Skip if only one mesh)

3. Decimate the merged mesh
   Select the joined mesh, add a Decimate modifier, and set collapse ratio to 0.15~0.2 to reduce face count to ~4k–6k.

Step 3: Export GLB model with Blender

1. Export GLB file
   File -> Export -> glTF 2.0 (.glb/.gltf)

   • Unselect: Transform -> +Y up
   • Select: Geometry -> Apply Modifiers
   • Unselect: Animation -> Animation and Shape Keys

2. Reduce texture resolution using gltf-transform:

 gltf-transform resize --width 1024 --height 1024 mixamo_Chad_merge.glb mixamo_Chad_merge.glb

Problem Shooting

A-1. Vertice joint > total joints

IndexError: index 194 is out of bounds for axis 0 with size 65

Solution: Open and re-save the GLB file with a 3D viewer like Gestaltor.

A-2. Mismatch between skin and motion joints

Solution: When downloading motion FBX on Mixamo, select a standard human character like Jackie or Michelle.

Load Avatars in Genesis

Parameters:

• motion_data_path: Path to motion .pkl

• skin_options:

  • glb_path, euler, pos

• ego_view_options: Optional

  • res, fov, GUI

• frame_ratio: Frame update ratio

• terrain_height_path: Optional auto-height support

Actions

Move forward

• move(distance[, speed=1])

Turn left

• turn_left(angle[, turn_frame_limit=15])

Turn right

• turn_right(angle[, turn_frame_limit=15])

Reach (Touch, Press)

• reach(hand, object)

Pick

• pick(hand, object)

Put

• put(hand[, position, rotation])

Throw

• throw(hand[, orientation])

Open

• open(hand, object)

Close

• close(hand, object)

Sit

• sit([object, position])

Stand up

• stand_up()

Drink

• drink(hand)

Eat

• eat(hand)

Play

• play(hand, duration)

Enter/Exit Vehicles

• enter_bus(bus)
• exit_bus()
• enter_bike(hand_id, bike)
• exit_bike(hand_id)
• enter_car(hand_id, car)
• exit_car(hand_id)

Other Functions

• reset(global_trans, global_rot)

• spare(): Return True if avatar is idle

  • from ViCo.modules.avatar.utils import ActionStatus

• get_global_xy()

• get_global_height()

• get_global_pose(): Returns [x, y, z, r, p, y]

• set_global_height(height)

• render_ego_view(rotation_offset, depth, segmentation)

Adding Your Own Motions

1. Download motion FBX into: Genesis/genesis/assets/motions/

2. Convert to pickle:

   python convert_fbx.py -e Genesis/genesis/assets/motions/ -o Genesis/genesis/assets/motions/

3. Add your module under: Genesis/genesis/assets/avartars/motions

Generate Scene-grounded Community of Characters

1. Prepare the OPENAI GPT-4o API key and create .api_keys.json under the root directory. Then, copy the following into .api_keys.json:

   {
     "all": [
       {
           "AZURE_ENDPOINT": "YOUR ENDPOINT HERE",
           "OPENAI_API_KEY": "YOUR KEY HERE"
       }
     ]
   }

2. Open scripts/generate_characters_all.sh and modify the scene(s) you want to generate characters, for example, if you want to generate characters for LONDON: SCENES=(LONDON)
3. Run bash scripts/generate_characters_all.sh to start generation process

Reminders

The script supports annotating multiple scenes at once, for example, SCENES=(LONDON TORONTO), but we recommend annotating one scene at a time.

Procedure Overview

There are 2 steps in the character generation pipeline. We provide an overview of main sub-steps of each step. Please refer to the source code for full sub-steps.

Step 1: Generate characters and annotate images of known places

1. Generate config-level place/building metadata
2. Sample Mixamo and celebrity characters from the pool
3. Generate character profiles and groups information with grounding validator
4. Annotate known places on the global image, which generates known_places.png

Step 2: Offset the xy positions for indoor places