One theory is that this could be due to your baby getting too much foremilk at the moment - the 'foremilk' comes at the beginning of the feed and it is watery and hydrating for the baby. As the feed progresses, the milk gets richer and fattier, and this helps fill the baby up and keep him full for longer. People call the fattier milk 'hindmilk' but actually there is no point where the milk switches from one to the other - the increase in fat content is gradual and progressive.
The watery milk contains high levels of lactose which can be hard for the baby's immature digestive system to process - this leads to gas and greenish poops. Some people worry about this, but it is actually not usually a problem, aside from the baby maybe being a little windy and uncomfortable (something pretty much all babies struggle with to begin with anyway). As long as the baby is gaining enough weight and producing plenty of wet nappies, you have nothing to worry about - ALL the milk contains some fat and it's perfectly possible for a baby to thrive while there is a milk imbalance.
As your baby is new, your boobs are probably producing absolutely loads of milk right now and you will likely be experiencing some degree of oversupply (nearly everyone does) - since your baby is still tiny, it only takes a little milk to fill him up, and therefore he may well be full before the milk has become fatty enough to produce a yellow poop - don't worry though, this doesn't necessarily mean he won't be gaining weight.
If he's fussy on the breast, especially at the beginning of a feed, this is possibly because of a fast and strong letdown of milk that is overwhelming him somewhat. If he comes off the breast and your milk drips very fast or sprays, you can be pretty sure this is causing his fussiness. If this is the case, you can take him off the breast when he begins to get upset and wait for the flow to subside before putting him back on, or try nursing in a reclined position so gravity is on your side. It should calm down as your milk production adjusts and he gets bigger and more capable of coping with the flow.
This happened to me around week 2/3 with my daughter and I was really worried about it, so I spoke to someone on the La Leche League forum and this is what they told me. My daughter gained loads of weight and within a couple of weeks she was doing yellow nappies again, though she still struggles with my letdown at times even now lol.
Hope this helps.