Side-channel vulnerabilities in software are caused by an observable imbalance in resource usage across different program paths. We demonstrate that just-in-time (JIT) compilation, which is crucial to the runtime performance of modern Java virtual machines (JVMs), can introduce timing side channels in cases where the input distribution to the program is non-uniform. These timing channels enable an attacker to infer potentially sensitive information about predicates on the program input. We define three attack models under which such side channels are harnessable and five vulnerability templates to detect susceptible code fragments and predicates. We also propose profiling algorithms to generate the representative statistical information necessary for the attacker to perform accurate inference. We first systematically evaluate the strength of JIT-based side channels on three widely used classes from the Java standard library: java.lang.Math, java.lang.String, and java.math.BigInteger. We then present examples of JIT-based side channels in the Apache Shiro security framework and the GraphHopper route planning server, and show that are observable over the public Internet.