Usually, a girl won't even consciously notice you're touching her if you're doing it right. If you're trying too hard to touch her, that's usually when she'll speak up.
How were you touching her? I usually like touching her arm with my elbow, kinda like a nudge sort of thing. Then i gradually nudge her more and more until I can keep my arm touched against hers. It's gradual and natural. How were you touching her?