The visitors took the lead when Jackson Jowett smashed home the goal of the game via the underside of the crossbar from 25 yards out with his weaker left foot.
Seamer levelled things up before the interval when a poor clearance from the Edgehill keeper landed at the feet of Ben Mason who launched the ball straight back over the gloveman’s head and into an empty net from distance.
Both sides had chances claim the ascendancy in the second period before the hosts struck through Gary Lawton who smashed home following a scramble after a corner.
With just eight minutes to go it looked as if Seamer were heading for the three points until substitute Aziz played in Jowett for his second of the day.
Aziz then finished off a fine one-touch passing move by calmly slotting into the bottom corner of the net with just seconds remaining.
Seamer boss Paddy Parke said: “I’m disappointed. A draw would have been a fair result.
“Wayne Aziz changed the game when he came on for them and we feel we should have had two penalties, one at 2-1 up for a blatant handball, then a second when Luke Delve was tripped in the last minute just after they had gone 3-2 up.”
Edgehill chief Steve Clegg agreed with his opposite number’s assessment.
“Seamer didn’t deserve to lose, we were very lucky to come away with the points,” he reflected.
“We’ve not been playing well in recent games, but we’ve still been able to get results.”
Champions elect West Pier took another step closer to the Division One title when they hammered Sleights 15-0.
The prolific Billy Logan led the way with six goals in a game that Andy Spivey’s men led 6-0 at the break.
Tom Barker, Will Jenkinson, Martin Cooper and Neil Thomas all netted doubles, and a goal from Chris Nelson rounded things off.
FC Rosette need just one point from their remaining two league fixtures to seal the third division title after a 2-0 success at second-placed Goal Sports.
Lewis Hird opened the scoring for the Newby side when he was sent clean through on goal by Joe Logan, the visitors taking a one-goal advantage into half-time.
Sports almost levelled but were denied by the woodwork before Logan netted with a neat close-range finish to seal the win after 70 minutes.
Rosette keeper Michael Harland was named his side’s man of the match after making a number of fine saves in the second period.
Itis Itis Rovers continued their climb away from the bottom of the table when they beat promotion-chasing Sherburn 3-0.
Ross Kevan gave the hosts a first-half lead after Dave Young got forward from right-back to tee him up.
Sam Pickard doubled the advantage after the interval when he embarked on a solo run from midfield and finished well.
Pickard then added a second from the penalty spot after Ellis Wilson was brought down in the box.
Niall Gibb, who started the game at left-back before moving to a central midfield berth, was Rovers’ star man