Counter Movement Jump (CMJ)
🎯 Goal
This is essentially the same as the first challenge. However, this time you start from a standing position and have to use a counter-movement in order to put yourself into a position where you can effectively jump. It sounds easy, but makes it quite a bit more challenging than the first one.
👨💻 Your Solution
📊 Results
🔒 Optimal Solution (Click to Reveal)
📊 Performance Comparison
👨💻 Your Results
🏆 Solver Results
📈 Your Performance
🧬 How Evolution Found the Answer
This challenge was much more involved to solve with the genetic algorithm. A very naive approach of defining the fitness function as the jump height resulted in the population to do a small jump using the plantar flexors only. As these individuals deemed more "successful" by the fitness function, they took over the population slowly and no real jumping strategy could evolve. So the algorithm got stuck in a local maxima.
The solution was to guide the evolution by preparing a "curriculum". The idea is to have different stages of the evolution where the fitness function changes slightly to encourage the population to learn different skills step-by-step.
The curriculum
- Phase 1 (Generations 0-40): Aim of the first phase was to teach the counter-movement only. To do this the fitness function rewarded crouch depth and knee flexion while punishing increased torso angle to the ground. Reason for the punishment was that without it the model would just throw itself down to get the maximum crouch score. You can see the evolution of this phase clearly in the first 40 generations where the model didn't try to jump.
- Phase 2 (Generations 40-60): This phase aimed to slowly introduce the concept of upward motion. The fitness function behaved exactly as the first phase with additional points for jump height.
- Phase 3 (Generations 60-150): By now the counter-movement is engraved into the population. For the last phase the fitness function focused solely on maximizing jump height. This didn't cause any regression since using only plantar flexion to do a small jump could never reach the heights achieved by using the counter-movement.
Fitness Function Code Snippets
Below is the simple fitness function I used for the squat jump challenge.
Since this function only checks the jump height the previous model didn't care what it did in the air as long as it achieved maximum height. Causing the random movements during the flight phase.
View the "Squat Jumper Fitness Function"
def get_fitness(self, json_data=None):
if json_data:
max_height = max(json_data['data']['center_of_mass_Y'])
jump_height = max_height - json_data['data']['center_of_mass_Y'][0]
self.fitness = jump_height
return self.fitness
return 0
And here is the messy Counter-Movement Jump Fitness Function.
View the "CMJ Jumper Fitness Function"
def get_fitness(self, gen_counter, json_data=None):
if not json_data:
return 0
self.fitness = 100 # Start from 100 to avoid zero fitness individuals
# Weights are tuned depending on the importance of each metric. They may seem arbitrary due to the different units (degrees, meters, etc.)
# It is basically trial and error to find a good balance
weights = {
'jump_height': 100,
'crouch_depth': 25,
'torso_deviation': 2,
'minimum_knee_angle': 2}
# This section is just extracting the relevant data. I was going to refactor it later. But you know how it goes.
com_y = json_data['data']['center_of_mass_Y']
torso_Ox = json_data['data']['torso_Oz'] #Torso medio-lateral orientation
knee_angle = json_data['joint_angles']['data']['knee_angle_r'] # Right knee angle
initial_height = com_y[0]
time = json_data['data']['time']
# I used timings to define phases of the jump. So the model wouldn't get punished for falling at the end of the jump for example
time_threshold_deviation = 0.5 # 0.5 seconds to assess torso deviation
threshold_index_deviation = next((i for i, t in enumerate(time) if t >= time_threshold_deviation), None)
time_threshold_crouch = .3 # 0.3 seconds to assess crouch
threshold_index_crouch = next((i for i, t in enumerate(time) if t >= time_threshold_crouch), None)
torso_deviation = max(abs(np.array(torso_Ox[:threshold_index_deviation]))) # Max deviation from vertical. Only consider first 0.5 seconds
min_height = min(com_y[:threshold_index_crouch])
crouch_depth = abs(initial_height - min_height)
minimum_knee_angle = min(abs(min(np.rad2deg(knee_angle[:threshold_index_crouch]))), 90) # Minimum knee flexion angle, capped at 90 degrees
jump_height = max(com_y) - initial_height
# Define training phases based on generation count
phase_1 = gen_counter < 40
phase_2 = 40 <= gen_counter < 60
phase_3 = gen_counter >= 60
#Early training phase (learn to crouch)
if phase_1:
self.fitness += crouch_depth * weights['crouch_depth']
self.fitness -= torso_deviation * weights['torso_deviation'] # Penalize torso lean to learn crouching upright
self.fitness += minimum_knee_angle * weights['minimum_knee_angle'] # reward for knee flexion
#Later training phase (Start rewarding jump height)
elif phase_2:
self.fitness += crouch_depth * weights['crouch_depth']
self.fitness -= torso_deviation * weights['torso_deviation'] # Penalize torso lean to learn crouching upright
self.fitness += minimum_knee_angle * weights['minimum_knee_angle'] # reward for knee flexion
self.fitness += jump_height * weights['jump_height'] # Introduce jump height reward
# Final training phase (Jump height is the ultimate goal)
elif phase_3:
self.fitness += jump_height * weights['jump_height']
return self.fitness if self.fitness > 0 else 0 # Ensure non-negative fitness