Follow links in llms.txt indexes when adding from a website (#66)