Simply, you don't have enough binders in there. In these we use quinoa and eggs, the mix is quite wet and after chilling holds together nicely on frying. If you need more binder, add a few breadcrumbs to the mix to bring it together, but these patties should not break up in the pan if made properly.