Filterdefinition is not working on drilldown top to bottom in category filter

The filtering works on top level, but once I drill down it seems to be switch over default filter
using fallback fallbackFilterdefinition