Reykjavik is pretty remote and it can be hard sourcing supplies. I have done the hard work so you don't have to - these are the places I rely on for my crafting needs
Map
The best place to get lopi yarn or an authentic sweater. They are all locally handmade.
Home - The Handknitting Association of Iceland
One of my favourite places for other miscellaneous craft supplies
Föndra - Alhliða föndurverslun með vefnaðarvöru, garn og föndurvöru
The best place for niche or specific art supplies, but beware of the restrictive opening hours
Föndurlist
Most affordable yarn, especially if you need something quickly. Also nice home decor
Jysk
Sometimes you just need industrial equipment (glue, wire, wood), imo Bauhaus has the best range of ... well, everything