Q:

how to force all internet traffic go through my vpn tunnel

I have my OpenVPN server running on my Linksys-E4200 router.  The VPN server local ip is 192.168.66.1.  Now I am in the public library.  My Windows7 OpenVPN client has connected with the OpenVPN server.  
When I use tracert to see the traffic for some unpopular websites on a DOS window, I got the following:
Tracing route to www.wenxuecity.com [35.190.55.229]
over a maximum of 30 hops:

  1    33 ms    29 ms    21 ms  192.168.66.1
  2    39 ms    32 ms    34 ms  10.16.0.1
  3   160 ms    30 ms    33 ms  bdle31-sub211.aggr1.sbo.ma.rcn.net [207.172.217.231]
  4    47 ms    43 ms   100 ms  bdle5.core1.sbo.ma.rcn.net [207.172.15.146]
  5    41 ms    45 ms    48 ms  hge0-2-0-0.core1.nyw.ny.rcn.net [207.172.19.211]
  6    57 ms    38 ms    40 ms  bdle4.border1.nyw.ny.rcn.net [207.172.15.69]

But when I try this for some popular websites, such as google, yahoo, cnn, I got this:
Tracing route to www.google.com [2607:f8b0:4006:811::2004]
over a maximum of 30 hops:

  1     5 ms     8 ms     3 ms  2603:3005:540b:7800:c02:27ff:fef2:eaa7
  2    27 ms    26 ms    30 ms  2001:558:4023:145::1
  3    13 ms    12 ms    13 ms  2001:558:202:406::1
  4    16 ms    15 ms    13 ms  2001:558:200:4020::1
  5     *       30 ms     *     be-7015-cr02.newyork.ny.ibone.comcast.net [2001:558:0:f6b6::1]
  6    35 ms     *        *     be-10381-pe02.111eighthave.ny.ibone.comcast.net [2001:558:0:f5be::2]

In the first case, I assume the client traffic is through the VPN tunnel to my home(OpenVPN server) and then goes to the destination. 
In the second case, these IPv6 addresses are from the local public library's network.  Does this mean that not all internet traffic from my machine/client are through the VPN tunnel? 
How can I force all the client traffic goes through the VPN tunnel first?  Is it a  client machine issue? 
EDIT:
After I have disabled the IPv6, I found I don't have any internet access if the vpn connection is established.  That means my vpn server doesn't allow me to go outside?   
By remote to my vpn server, I can see two vpn connections, one from my laptop and one from my phone.  But there is nothing received for both of the connections. 

A:

As Linksys E4200 stock firmware doesn't support OpenVPN, you probably have DD-WRT.  Your issue is not likely caused by the router's configuration, but the specifications of your home Internet. 
The difference between www.wenxuecity.com [35.190.55.229] and www.google.com [2607:f8b0:4006:811::2004] is not the popularity but the Internet Protocol version used: www.wenxuecity.com doesn't have AAAA record and is only served on IPv4, while all the big sites you tested have both IPv4 and IPv6. 
Your public library has IPv6 connectivity, which either your home Internet connection or your OpenVPN setup lacks.  This means that you don't have a default IPv6 route through your OpenVPN.  You can check your routing tables using route print to verify. 
If your ISP doesn't support IPv6, this can't be fixed. 
One option is to disable IPv6 on your Windows 7 machine.  From GUI:

From Network and Sharing Center
select Change adapter settings
right-click the correct adapter and select Properties. 
Uncheck Internet Protocol Version 6 (TCP/IPv6). 

It's also possible to disable IPv6 on all non-tunnel interfaces from Windows registry:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters]
"DisabledComponents"=dword:00000010

Using dword:000000ff disables IPv6 altogether.  
(See Guidance for configuring IPv6 in Windows for advanced users.)