The most important criteria for a startup is access to great staff. And that means you are located in a neighbourhood with decent transportation links. A lot of startups are located in the downtown core (bounded by Bathurst on the West, Jarvis on the East and Bloor on the North). This also has the added benefit that its easy for teams to socialize after work as there is a vibrant restaurant and bar scene in the downtown core. With all the startups, there is also a lot of meetups each night. It all becomes self-perpetuating with each benefit feeding into the other.
Having said this, you can make a startup work in other places, its all about getting the right team together. Some companies prefer to be a bit away from the core as it’s also fairly competitive for hiring. Best of Luck.