This is a not-uncommon feeling for Lisp-like languages, and if you want a benchmark on a particular problem I'll point to http://shootout.alioth.debian.org/u64q/benchmark.php?test=fa... where the "alternative" Common Lisp program is the fastest. (Just ahead of a Java 7 program.)

Why does the feeling persist? Because there's a lot of anecdotes and personal experience by lispers who have created something they didn't think they could do in another language "from scratch". I'll offer a link to a different anecdote: http://www.cs.indiana.edu/~jsobel/c455-c511.updated.txt "Is Scheme Faster than C?" Here are the highlights, with the most important quote at the end.

…One of the assignments in the
Algorithms course was a major term project in which we were to
implement and optimize the Fast Multiplication algorithm. Fast
Multiplication is a recursive "divide-and-conquer" algorithm for
multiplying two numbers, especially large numbers: hundreds or
thousands of digits.…Part of our grade for this project was based on how fast the program
ran.

…What I noticed was that each of those
three recursive calls had nearly the same control context. In a
simple-minded implementation in C, that context would be saved and
restored three times, being destroyed after the return of each call.
I thought to myself: If only I had explicit control over the flow of
my program, I could speed it up significantly by creating that context
only once and using it three times, destroying it only after the
return of the third recursive call. Function calls are so costly!
But I would never want to attempt such a thing from scratch in C.

…Furthermore, I could start with a simpler, more naive program and
basically DERIVE the sophisticated one through a series of
correctness-preserving program transformations. This is where Scheme
really won. Because of its extremely algorithmic---almost
mathematical---nature, Scheme can be easily manipulated in a sort of
algebraic style. One can follow a series of rewrite rules (just about
blindly) to transform a program into another form with some desirable
property. This was exactly what I needed.

…I had never had the courage to
use a "goto" in C before; it was just too dangerous a tool. It was
kind of fun to use them now, knowing that I was completely safe in
doing so. That was the amazing part: I had PRODUCED a program that I
could not have WRITTEN, and would not have wanted to write directly,
for that matter.

…"But does it run
fast?" Speed was, after all, my main goal. The answer is a very
resounding "Yes!" Out of a class of about 15 students, only one
person beat me (and only barely), and he wrote significant portions of
his program directly in Assembly Language.

…Real efficiency comes from elegant solutions, not
optimized programs. Optimization is always just a few
correctness-preserving transformations away.

Lisp-like languages make correctness-preserving transformations easy with their homogeneous syntax and accompanying macros. It may be that a simple Java implementation of the GP's example would be faster, I'd like to see a benchmark too, but charged with the task of making each one faster relative to itself, I know I'd rather work with the Clojure code because it's easier to transform. Compared to this http://mechanical-sympathy.blogspot.com/2012/07/native-cc-li... on the problem of object serialization, getting really fast Java takes a lot of work whereas getting pretty fast Clojure takes a lot less work (if the parent post is to be considered a "pretty fast" implementation of the particular data processing problem).