Agnew played a full 90 with a pulled hamstring, while midfielder Pete Davidson was forced off at half-time with his persistent hamstring injury and defender Benn Lewis also battled through the whole NPL East game with a groin strain.
Agnew said: “We were decimated by injured with myself, Benn and Pete all having to start the game with injuries.
“We were also further weakened when Danny Earl went off with a finger injury when he fell over and a Taddy player stood on his hand.
“He is going for a scan this week then we will find out how bad it is, but that is just our luck at the moment.
“We were also without the suspended Nathan Hotte as our appeal to the FA regarding his red card against Ponte was rejected despite us having clear video evidence of the perpetrator being the visiting player and not Nathan.
“Missing Nathan for four games is a huge blow, but the problems with the balance of the team started as soon as Grimsby recalled Jaz Goundry from his loan spell as he and Nathan formed a solid partnership in central midfield.
“It is no coincidence that our form has dipped since they have not been available.”
Agnew admits: “Tadcaster deserved to win but it was so frustrating we were decimated by injuries as a win would have put us nine points clear but a loss left us still in trouble only three points clear.
“I have been on the phone to 15 pro clubs looking to bring players in on loan but we just cannot bring them in.
“It continues to be frustrating that Grimsby recalled Jaz but he has not been in the first-team squad as he would have been playing matches with us.
“But with loan players this is always the parent club’s right to recall players, and Grimsby had been struggling when they recalled him.
“As it stands I will have to play again on Saturday at Hebburn with injury, I am not sure about Benn but he did a great job playing on with an injury Saturday to help the team out.
“As for Pete (Davidson) it looks like he could be out for a fair amount of time.
“On Saturday Hebburn, who are only three points ahead of us, will be a tough game .
“We’re in for a very difficult run of games after this, we host Worksop on February 5 then play at second-placed Marske the following Tuesday then are at home to leaders Liversedge on February 12.
“I could really do with getting some players in for that trio of games in particular!”
On Saturday, following a spell of pressure and chances, Eddie Church opened the scoring for Tadcaster when he tapped home a rebound after a good save from Town keeper James Hitchcock three minutes before the break.
Church got Tadcaster’s second goal after 55 minutes with a brilliant strike. After 74 minutes, the linesman judged Donald Chimalilo’s scrambled effort to have crossed the line.
Mark Ferguson’s shot deflected off a defender and into the net to seal the 4-0 win.