For many years, I used a fine wire wheel for aluminum electronic cases that I manufactured. It was 1" wide and mounted on a 1/3 hp. 1725 rpm motor. Over the years, I brushed many hundreds of cases.
It is important not to have any contamination on the surface being brushed or the wheel as it will create a shiny surface. The wheel "bites" into the surface with an effect similar to the galling observed when machining aluminum without a lubricant. I would start at the middle, keeping the grain running parallel brushing to the edge and flip the part to repeat the process to the other edge. Too much pressure would start to melt the surface creating irregular patterns so some care was needed.
The brushed cases were then sprayed with a clear acrylic to prevent fingerprints from affecting the fresh surface.
For larger matte surfaces, I used an orbital sander which left a swirled pattern. You can control the pattern with the grit of pad you use. Again grease, oil, etc., no matter how little, is your enemy. I would wash the surface with Dawn detergent, rinse well and an acetone wipe to remove all traces of oil. I used this process on 15" x 60" sheets.
Bob