Here is a table I use to estimate the output differences as you step up in the size of woofer and from sealed to bass-reflex enclosures. This will be based on the same series of woofer and may be a little different from brand to brand or series to series. But you'll get the basic concept.
10-inch sealed 1.04 cu.ft. ext. encl. Output of 84 dB 1w/!m. Half power @ 47 Hz.
12-inch sealed 1.86 cu.ft. ext. encl. Output of 86 dB 1w/1m. Half power @ 41 Hz.
10-inch bass-reflex 1.65 cu.ft. ext. encl. Output of 87 db 1w/1m. Half power @ 38 Hz.
12-inch bass-reflex 2.72 cu ft. ext. encl. Output of 89 db 1w/1m. Half power @ 33 Hz.
---- and so on ----
The enclosure displacement is figured as external so you can immediately determine if you have the space with legroom, etc. for the particular sub package.
As you can see from the table past a point the enclosure has a profound impact on the deep bass extension. A bass-reflex 10 may have more low bass contribution than an optimized sealed 12 enclosure. You definitely do not want to place a larger woofer in an undersized enclosure.
To put this in perspective a difference of +3 db is the equivalent of doubling your amplifier power. So going from a sealed 10 to a bass-reflex 12 is almost four times the power. Plus the difference in deep bass extension is dramatic.
As for amplifier power you just need to have a good match between the subwoofer and amplifier. In an open field boat environment power doesn't go nearly as far as it would in a vehicle so as a general rule its not a bad idea for your amplifier power to be right at the upper end of the woofer's thermal or continuous capacity. Overmatching either your subwoofer or amplifier is a mistake. Balance is important.
There are other considerations and always exceptions but this simple and general table should serve you well.
Start with the available space for your enclosure, or what you are willing to sacrifice. The boat will determine the maximum enclosure external displacement which will in turn determine the best enclosure loading type and best size of woofer. In a sense this places the horse before the cart.
Thanks David there is a ton of great info in that post.