Vienna has many facilities for the dog owner. Many shops will have a "Hundparkplatz" outside... a small hook or two to tie your dog's leash to while you shop. Most parks will have a "Hundzone": a fenced-in off-leash area for dogs to play and meet. Public transit is dog-friendly, but you will be expected to keep your large dogs on a leash and to have their mouth covered with a "Beißkorb" (dog muzzle), which older dogs may have a hard time getting used to.
Now that you have some relevant terms you should have no problem googling for more information (for example Hundzones should be marked on the official City of Vienna online map at wien.gv.at).