No. This is a common misconception. Although different parts of the brain are more or less active during different activities, there is no evidence that we use only a small portion of our brains. Damage to a small area of the brain can cause devastating effects, such as amnesia, paralysis, or loss of language. This suggests that every part of the brain serves an important function, upon which other parts of the brain depend.
On the other hand, some people— especially children—can recover after suffering major damage to the brain or even losing part of it. Such remarkable recoveries do not suggest that we need only a fraction of our brain, however. Rather, they illustrate the tremendous capacity of the brain to "rewire" itself: Cells in the remaining parts of the brain form new connections and take over the functions of those parts that were removed.
