I had the pleasure of demoing a Specialized Turbo Levo SL this weekend. It was awesome. I couldn't stop smiling. My only complaint was that I wanted to keep riding it, but it was time to go home. That is really the only reason I wouldn't consider owning one as my only or primary bike. If I was in a situation that I wanted to ride longer or couldn't keep up with the pace and distance of the people I regularly ride with, I'd strongly consider it, but for my average allotted window of time I get to ride, I like the exercise the traditional bike gives me.
We rode in a group of 2 e-bikes and 3 regular bikes. There were a few sections of tight, twisty, or technical that the regular bikes were faster. The e-bikes were definitely faster on the extended climbs. Overall, the group still stayed together just as well as any group of 5 average riders of varying fitness. It takes the same level of bike handling, if not more, to keep the wheels moving in the right direction with the extra weight. Technical sections were still challenging.
The bike I rode was around 40lbs., so not too bad really compared to some 35+ lb pedal bikes on the market. Compared to other e-bikes discussed in this thread, it's about 10 lbs lighter. (And this was the lowest spec'd Comp version.) I could definitely feel the added weight on jumps and techy stuff when I was trying to unweight the rear end, but it wasn't anything that I couldn't deal with. I rode it some with the assist turned off. It wasn't "fun", but if there was a motor failure, it would be completely doable to pedal it out.
Even though it was the SL, thus smaller motor - less power, it had three levels of assist. The strongest "turbo" was too much for normal riding. I was fun zooming uphill on the road or a stretch of double track. Most of the time, Eco mode was all that was needed, but the middle setting was nice at times too. It really is not a set it and go. You still need to shift gears and plan for a climb.
Also while we were out there, we ran into a group of 5 or 6 riders, all probably mid-20's, all riding eMTBs.