I can highly recommend reading the book "Never Eat Alone". I read it years ago but still think about it regularly. One thing I remember from the book is that the author suggest you talk to a stranger at least once a day. It is a lot easier than you might think. The first week is awkward and you will be nervous a lot. After that you will find that people enjoy it immensely if given an opportunity to talk to someone.
To answer your question more specifically; it really doesn't matter what you say and it can be the smallest thing. Like "Busy huh?", "How are you doing?" or just "Nice coat" and people will either just nod and then ignore you (nothing lost) or jump at the opportunity and start talking.
Try it for a week and let us know how it worked out!