A downside to double blind review

A discussion started by Daniel Lemire reminded me of this issue. I recently participated in refereeing for a CS conference using double-blind review. I noticed an issue that I have not seen mentioned before (for example in the IMS Ad Hoc committee report). Several papers were not accepted, but they had some good ideas. If I now write a paper building on these ideas, I have no idea whom to cite or credit. I suppose I could ask the programme chair to put us in contact, but of course that may not occur for a long time (especially given the speed at which I can write papers these days). Still, this seems an obvious drawback not shared by single-blind review.