Reykjavik is pretty remote and it can be hard sourcing supplies. I have done the hard work so you don't have to - these are the places I rely on for my crafting needs
Map
The best place to get lopi yarn or an authentic sweater. They are all locally handmade.
Home - The Handknitting Association of Iceland
One of my favourite places for other miscellaneous craft supplies
Föndra - Alhliða föndurverslun með vefnaðarvöru, garn og föndurvöru
The best place for niche or specific art supplies, but beware of the restrictive opening hours
Föndurlist
Most affordable yarn, especially if you need something quickly. Also nice home decor
Jysk
Sometimes you just need industrial equipment (glue, wire, wood), imo Bauhaus has the best range of ... well, everything
BAUHAUS - Eitt mesta úrval landsins
Small but handy craft store
Litir og Föndur
A good place for paints and crafts as well as carpentry supplies
Slippfélagið
A bit of a stretch but this is the best place to buy silicone spray (for wigs)
