The York League opened the scoring on 26 minutes through J Mackie, but Itis Itis levelled just four minutes later through a near-post header from Jack Ramos.
There were no further goals and the final went to a penalty shoot-out, which was won 5-4 by the York League side.
Rovers manager Jordan Lee said: "They wanted it more. Our preparation probably wasn't good enough, only had one friendly and it showed after 70 minutes, we looked leggy.
Scarborough Athletic's home National League North campaign off to a winning start against Hereford
RETRO SPOTLIGHT: 15 nostalgic Scarborough photos from the Scarborough News sports archives
Ebberston battle back to sink hosts Filey in SBL Premier Division thriller
Scarborough Athletic slip to 2-0 defeat at Brackley Town in National League North opener
Former Harrogate Town and Bradford City goalkeeper Joe Cracknell returns to Scarborough Athletic
"Credit to them they pushed us to the end, and there was two great saves from their keeper in the shoot-out.
"Credit to my whole back four, Curtis Rose and Callum Gravestock in particular. Absolutely outstanding effort from them two.
"These things happen in football I'm afraid, we just have to pick ourselves up and get ready for the new season."