Metallurgy 101, with sliding surfaces one must be softer than the other otherwise galling and or microwelding will take place. The stock stamped steel rockers were softer material than the shafts. With the comp stainless rockers they are harder than the shafts. Ok, with some mild spring pressures. Will destroy the shafts with killer springs (not the level used here).
The fix is to have them bushed with bronze inserts which adds $ and thins the steel section, both not good.
Aluminum rockers are good but will fail eventually due to work hardening, should never be a issue with hydraulic cam spring pressures. Excellent aluminum rockers such as T&D almost never fail even with 600+ pound valve spring pressures, but prepare to use your highest limit credit card.
Moral of the story is...
You can't put 2 hard metals together under higher loads with intermittent oiling. A failure will result.
To the OP. You should just put stock type stamped rockers with that cam. I stand by my original diag of long pushrods or clogged supply holes