Originally, this term simply referred to areas (on civilized Core worlds) entirely 'off' the Cortex network (aside from illegal pirate feeds) -- as time progressed, a person's non-existence within the 'legitimate' world of Credit and Cortex became a crime in and of itself1 and Blackout Zones became synonymous with havens of crime, moral turpitude, and treasonous anti-government activity. For an honest citizen of the Alliance, merely being in a Blackout Zone is enough to warrant arrest, as it is assumed that one would not associate with the denizens of such an area if one did not intend to engage in illegal activity.
(Blackout Zones are mentioned in the flashbacks to Simon Tam's pre-Serenity days in the episode Safe; Simon enters a Blackout Zone in order to contact people who might be able to help him get River out of The Academy.)
1 - Note here the beginning of a connection between being 'good' and being 'connected'. In a very real sense, not being connected to the Cortex (and, thus, the Alliance) goes hand-in-hand with being a Bad Citizen. From this, one wonders how the Core world citizens must see settlers on the colonies.