Hilary Duff arrives at the gym for an afternoon workout