From PyPy's blog: Adding a JIT to your interpreter
When it (the jit compiler) detects a loop of code in the target language that is executed often, the loop is considered "hot" and marked to be traced. The next time that loop is entered, the interpreter gets put in tracing mode where every executed instruction is logged.
When the loop is finished, tracing stops. The trace of the loop is sent to an optimizer, and then to an assembler which outputs machine code. That machine code is then used for subsequent loop iterations.
(The generated machine code) depends on several assumptions about the code. Therefore, the machine code will contain guards, to validate those assumptions. If a guard check fails, the runtime falls back to regular interpreted mode.
This is one of the best explanation of JIT compilers I've found. I finally understand how they work after hearing the term tossed around so much lately.