In case anyone is interested (and has some ability to read Polish aktas), there are a large number of Jewish BMD records from the period 1808 to 1825 (i.e. pre-1826) within the "Roman Catholic" (actually should be called "civil") record set at the Polish archives.
These can be viewed here: [https://szukajwarchiwach.pl/35/1644/0/?q=belzyce+XSKANro:t+XARCHro:...]
i.e. just search the records for Hebrew signatures, or look in the indices at the back for "starozakonny".
To the best of my knowledge, nobody has ever tried indexing these records for Belzyce, although similar records have often been extracted at JRI Poland for other towns.