Rocket Landing Simulator

Gate Check #4 - Artificial Intelligence
Go Back to the Assignment Description

Part 1 (Overview) - Part 2 (Getting Ready) - Part 3 (AI Logic Part I) - Part 4 (AI Logic Part II) - Part 5 (AI Training)

Let's Learn How to Land

You are almost finished ! Hang in there!

For our final trick, we will write some code to allow our AI to "learn" how to land on the rocket. If you recall from the previous section, we provided you with a module called ga_helper. This module contains a very simple genetic algorithm. For a good introduction to Genetic Algorithms, click on this link. Here, however, are the highlights:

Don't be scared! The genetic algorithm has already been written for you (i.e., you don't have to add any code to ga_helper)! All we have to add some code to our application to hook the GA up to our application. Fortunately, this process is relatively straightforward. The first thing you need to do is open rocket_ai_[YOUR NAME]. Then, add the following line to the top (just below the import statement). 

# Initializes the AI Population
ga3_helper.initialize_genetic_algorithm(2, 1, 10.0, 10.0)

This function call activates the Genetic Algorithm, and passes it an initial set of parameters. If you look at this line in detail, you will see that initialize_genetic_algorithm() takes 4 parameters:

  1. The number of members to keep from each generation
  2. The number of "offspring" to generate after each generation has ended (this, combined with #1, must be less than the # of times you allow a scenario to be played before generating a new one)
  3. The maximum allowed velocity in the x direction (i.e., the x speed limit)
  4. The maximum allowed velocity in the y direction (i.e., the y speed limit)

For now, let's keep these values. We will be adjusting them later.

The next thing we need to do is to make sure that our run_autopilot() function uses the max x/y velocities and thruster amounts generated by the GA. To do this, go to this function and modify it as follows: 

def run_autopilot(run_number, rocket_x, rocket_y, rocket_vx, rocket_vy, ROCKET_WIDTH, landing_pad_x, landing_pad_y, LANDING_PAD_WIDTH):

    # Get the parameters from the genetic algorithm
    1.  ai_parameters = ga_helper.get_ai(run_number)
        NOTE:  ai_parameters now contains tuple containing (in order):
        - THRUST_AMOUNT_X
        - THRUST_AMOUNT_Y
        - MAX_X_THRUST
        - MAX_Y_THRUST
    
    2.  Set the following variables in your function to the values returned from ga_helper.get_ai()
        - MAX_Y_VELOCITY  = ai_parameters[3]
        - THRUST_AMOUNT_Y = . . .
        - MAX_X_VELOCITY  = . . .
        - THRUST_AMOUNT_X = . . .
    
    3.  THE AUTOPILOT CODE YOU WROTE FROM THE PREVIOUS PAGE GOES HERE

The final thing we need to do is modify our analyze_results() function so that it sends the score you generated to the Genetic Algorithm. The algorithm will then use these values to determine which members of the "population" (i.e., which parameter combinations) to keep.

To do this, go to analyze_results() and insert the following line of code once you have calculated the score for a particular run. 

def analyze_results():
    # Write some code to determine how you want to score your AI

    # Gives your score to the genetic algorithm
    ga_helper.score_ai(, )

Training

To training the GA, run your application and look in your IDE's console. If the GA is running, you should see something similar to the following:

Each of the "AI #" lines represents a set of thruster values. Your program will test each of these values on the same scenario. At the end of each run, your algorithm will send the score back to the GA. In this particular example, my program gave AI 0 a score of -2000 because it crashed into the water. Meanwhile, it gave AI 1 a bad score because it landed on the boat, but took too much time.

From this point on, it is up to you to play with the genetic algorithm in order to train a useful AI. There are two strategies you can take.

STRATEGY #1: Update Your Scoring Algorithm

When we coded analyze_results() in Gate Check 3, we created a very simple scoring algorithm that assigned points strictly based on whether or not the rocket landed on the boat. In order for the GA to work, however, we need to have a more fleshed out formula for scoring the rocket's performance that takes into account each of the performance metrics that we care about.

IMPORTANT: There is no "right/wrong way" to do this. Basically, you need to come up with a way to quantify the "goodness" of a landing. For example, let's say that you wanted to assign points based on how quickly the rocket landed on the boat. One way to do this is to start off with a base number of points, and subtract the number of animation frames that it took for the rocket to land on the boat, like so: 

speed_points = 1000 - time_elapsed_in_frames

This equation would give more points to an AI that landed in 200 frames than one that landed in 500 frames.

Once you have created an equation for each chacateristic, you can then assign a weight value to each one. For example, you might decide that speed is more important than the amount of fuel consumed, so you could assign the former a weight of 0.3 and the latter a weight of 0.1. You could then compute the "composite score" by: 

composite_score = speed_points * speed_weight + fuel_points * fuel_weight + . . .

As you can see, there is a lot of grey area. Experiment with various equations until you find one that works for you.

STRATEGY #2: Give the Genetic Algorithm Different Parameters

Earlier, we showed you how to initialize the genetic algorithm with a default set of parameters. However, you are free to try different values.

There is no perfect set of parameters. Instead, you just have to try a bunch and see what works.

Testing

Run your simulator for a few hours in order to give the AI time to train on a wide range of scenarios. When you are done, you will need to look at the console output to find the "best" AI you trained. It will look something like this:

Extract the x/y thrust and max speeds from this line. Then go back to rocket_ai_[YOUR NAME] and uncomment the following line of code. Make sure that you replace each parameter with the value from your best AI! 

# --------------------------------------------------------------
# This line sets the AI you want to submit to the competition
# When you are done with the assignment, uncomment this line and
# replace the parameters with the values of your "best" AI
# --------------------------------------------------------------
ga_helper.use_ai_configuration(HORIZONTAL_THRUST_AMOUNT, VERITCAL_THRUST_AMOUNT, MAX_HORIZONTAL_SPEED, MAX_VERTICAL_SPEED)

This line tells your GA to stop learning, and to only use the AI parameters you specified.

Once you have completed this step, you have finished the assignment. Congratulations!