Time Out says
If you're looking for fresh seafood, head to Stearns Wharf in the early morning hours to wait for the fishing boats to unload their catch. You can buy straight from the boats and take your fish home with you on ice—it's cheaper, fresher and way more fun. Avoid the pier during the afternoon hours, when it becomes a tourist trap Although the pier (home to California's oldest working wharf) offers wine tasting, a touch quarium for kids and lots of shopping and dining options, avoid it after mid-morning—it tends to become a bit of a tourist trap as the day goes on. Tip: If you're picking up fresh catch, don't miss Santa Barbara's famous uni and spotted prawns.