The War Galleon is physically bigger, and has larger cargo space and more broadside firepower. They are well suited for cargo runs and solo raids. They cost 40,000 gold.
The War Frigate has more deck cannons (including front facing ones), has more health and is generally more suited to crew operations. They cost 60,000 gold.