The second division side took the lead after 15 minutes when Finn McCusker's through-ball found Paul Mills and the striker fired home from 16 yards.
Tavern then doubled their lead 10 minutes later, Mills providing the pass through and player-boss Dan Thomas challenged Ayton keeper Andy Roebuck and beat him to the ball, sliding the ball into the net.
Ayton pulled a goal back five minutes before the interval when keeper Jordan Wood punched the ball into his own net, and the villagers then levelled after the interval through a second own goal.
A combination of missed chances by Ayton and solid keeping from Wood meant that the score remained at 2-2 so the game went to extra-time.
In the first minute of extra-time Mills was sent off for showing dissent when the referee ruled out his goal for offside, and more poor Ayton finishing and solid defending and keeping from Tavern took the tie to penalties.
All five penalty-takers, Kyle O'Toole, Wayne Shaw (who after coming on as a second-half sub in normal time suffered a broken toe but played on), Dan Blanchard, McCusker all fired in their spot-kicks, striker Ryan Baldry's effort, Ayton's fifth, was saved which left the scores locked at 4-4 with only Thomas left to take Tavern's final penalty and he slotted home to take his team into the quarter-finals.
Thomas said: "Duncan McDermott was our man of the match, he was everywhere in central midfield, he may be the oldest player in our team but he never stopped throughout the game.
"I am really happy with the win, Jordan Wood made some great saves for us but all the lads did really well."
Ayton player-boss Mark Plumpton added: "We had plenty of chances today but our strikers had an off-day. Jake McAleese was our man of the match at centre-back."
Newlands had to work hard for a 3-2 home win against Roscoes Bar.
Roscoes went ahead after only five minutes with a 35-yard strike from Jamie Wray which looped over the Newlands keeper and into the net.
Ten minutes later the visitors lost possession just outside their penalty area and Stew Bates made them pay with a neat finish.
The hosts, who had been forced to draft several youngsters into their team from the reserves then went 2-1 in front when Hughie Brown headed in from a corner.
Roscoes battled away and had one effort cleared off the line and blazed two shots over the bar.
Brown's second headed goal of the game looked to have secure a comfortable win for Newlands on the hour mark, but Bates went off with a knee injury and Roscoes came back into the game, scoring a second when Graeme Farrah played Raul Aldea through and he beat two men outside and then cut inside and shot into the bottom corner.
Roscoes pushed for the last 15 minutes looking for a leveller but Newlands defended well to keep them out, the hosts surviving a late penalty scare when Wray seemed to be brought down in the area.
Michael Wilson was man of the match for Newlands, with Liam Rowley, Martin Hadrick and Craig Spooner standing out for Roscoes.
Angel claimed a 5-0 home win against second division outfit Hush.
The hosts went ahead when Danny Collins volleyed in a Joe Gallagher cross and doubled their advantage when Martin Cooper applied a neat finish.
The away side were hampered by several no-shows leaving boss Dan Sheader with only one sub, and their troubles increased when Benny Davis added a third early in the second half.
Cooper's second clinical strike wrapped up the last eight spot for Angel, Davis smacking home his second of the game from the edge of the box to complete the scoring.
Left-back Dan Marsden, the starting point for a lot of fine moves, was named as the Angel man of the match, while central midfielder Liam Mintoft was the star man for Hush.
Trafalgar eased into the quarter-finals with a 9-0 home win against Division Two side Ayton Reserves.
Danny Glendinning and Alex Sheader banged in a hat-trick apiece for Col Jenkinson's troops, with Grant Elwell, Sam Pickard and a 25-yard free-kick from Liam Vasey completing the rout.
Josh Young and Jamie Patterson shared the man of the match honours for Traf, with keeper Jamie Taylor and centre-back Leroy Donaldson the star men for Ayton's second string.
The game between West Pier and Fylingdales was abandoned after 45 minutes due to adverse weather conditions with the scores locked at 0-0.
Angel Reserves surged into second spot in Division Two with a 5-0 win at Newlands Reserves.
Dom Barber opened the scoring for the visitors in the first half from the penalty spot.
Mike Hernandez came on at half-time and he added a second, before three late goals Robbie Scarborough and Macca Youngson (2) secured their passage into the quarter-finals, his second goal a powerful header from a corner.
The man of the match for Angel was left-back Luke Jenkinson.
Newlands' man of the match was Connor Bell, while Mikey Anderson also impressed for a depleted home side.
Fylingdales Reserves powered to a 14-0 win at home to eight-man Golden Last.
Charlie Smith, Jack Gale and Lewis Peel scored a hat-trick apiece for Dales, with Sam Russell (2), Nick Craven, Dylan Goldsmith and Taylor Humble also notching.
Right-back Craven was named as the man of the match for the victors.
Golden Last player-boss Liam Bowman said all eight players deserved the man of the match award for battling on in awful conditions with such a depleted side.
Valley eased to a 7-2 win at Radio Scarborough.
The game started in exciting fashion as Valley sliced through Radio Scarborough in the second minute with quick passing play before Sam Westmoreland smashed the ball into the top corner from 20 yards after beating his man with ease.
A long pass from the hosts' Ryan Hunter then caught the wind in the 13th minute and lifted over Valley keeper Stefan Campbell.
Valley replied two minutes later as they won a corner and Westmoreland finished from close-range.
Veteran striker Phil Clothier then went through on 25 minutes and chipped the keeper to further show the Valley dominance.
After the break Valley got off to a flying start again as Westmoreland picked the ball up from range and bent the ball into the top corner from 30 yards, completing his hat-trick on 48 minutes.
After a long passing move 10 minutes later, Valley carved open Radio again as Westmoreland flicked the ball around the defender for midfielder Brad Marshall to run on to and finish nearly past the keeper.
Dane Robinson then lobbed home from close range to earn the hosts a second goal.
Valley soon replied though as winger Danny Malton bent a 25-yard free-kick into the top corner on 70 minutes.
Five minutes later Paul Nuttall finished nicely in the box to make the fiinal score 7-2 to Valley.
Valley boss Ben Kristensen said: "I'm very happy about the drive and commitment of all my team today. Excellent finishing and play.
"Man of the match was Westmoreland, for his three goals,including two long-range screamers, and two assists."
Billy Sullivan was named as the home team's man of the match.