Fear can be a powerful motivator. It can also be a powerful roadblock and often stops people from realizing success. Could be fear of failure, fear of being seen as crazy, or fear of being judged by your friends/family, but fear almost always plays a role in not getting started.
Ask yourself this: What's the worst that could happen?
If you have the skills to start a successful business you probably have the skills to get a decent paying job. So if all else fails you get a job... Doesn't sound very scary. Maybe people will see you as crazy and your friends and family will judge you a little bit. Who cares? We've already talked about how the most successful people on the planet are often seen as crazy.
Don't let fear get in the way of creating the life that you want. The worst possible outcome really isn't that scary. The life you've been dreaming of is often on the other side of the thing you are most afraid to do.