In what form? As the primary ingredient in a commercial food (like salmon and rice flavor ), or just raw fish, or even cooked fish, as the only food item? If you're asking about just feeding fish and only fish then no, that would not be a good diet as it would be very deficient in a lot of vitamins and minerals that your dog needs. As a flavor in a commercial diet sure.
Did you really mean he's 2 weeks old? If so then his only diet should be puppy milk!!