The “SEO-competition” is basically about having a lot of keyword-targeted interlinked text and inbound links from authority sites. Having the best site in the world for downloadable content won’t help unless people are actually able to find your site in the search engines.
Odds are they won’t, since competing sites with way less downloadable content likely have hired writers to scribble out thousands of “how to find find free downloadable poetry”, “the top ten sites for downloading great open source games” etc. All such articles give ample opportunities to link to other sites, many of the site owners will out of self interest tweet about those articles, linking back to them, increasing the site authority of your competition. And winning the SEO-competition.
You can also hire a bunch of writers to create such articles, and consider it a marketing cost. But first of all I suggest creating rich descriptions about each downloadable content piece. An image can say more than a thousand words they say, consider therefore to put down 1001 descriptive words for each image they can download. This must be done for each locale/language you would like to rank for.
Another thing you can try is to hire virtual assistants to do outreach to bloggers. If you offer some kind of incentive, like premium membership for a year, many will mention your site favorably and give a non-sponsored link to your site.
Once you manage to pull in enough people, positive usage metrics can help boost your site further up the results ladder.