Scrapy 1.4 allows remote attackers to cause a denial of service (memory consumption) via large files because arbitrarily many files are read into memory, which is especially problematic if the files are then individually written in a separate thread to a slow storage resource, as demonstrated by interaction between dataReceived (in core/downloader/handlers/http11.py) and S3FilesStore.
The product does not properly control the allocation and maintenance of a limited resource.
| Name | Vendor | Start Version | End Version |
|---|---|---|---|
| Scrapy | Scrapy | 1.4 (including) | 1.4 (including) |
| Python-scrapy | Ubuntu | artful | * |
| Python-scrapy | Ubuntu | bionic | * |
| Python-scrapy | Ubuntu | cosmic | * |
| Python-scrapy | Ubuntu | disco | * |
| Python-scrapy | Ubuntu | eoan | * |
| Python-scrapy | Ubuntu | focal | * |
| Python-scrapy | Ubuntu | groovy | * |
| Python-scrapy | Ubuntu | hirsute | * |
| Python-scrapy | Ubuntu | impish | * |
| Python-scrapy | Ubuntu | kinetic | * |
| Python-scrapy | Ubuntu | lunar | * |
| Python-scrapy | Ubuntu | mantic | * |
| Python-scrapy | Ubuntu | oracular | * |
| Python-scrapy | Ubuntu | trusty | * |
| Python-scrapy | Ubuntu | xenial | * |
| Python-scrapy | Ubuntu | zesty | * |
Mitigation of resource exhaustion attacks requires that the target system either:
The first of these solutions is an issue in itself though, since it may allow attackers to prevent the use of the system by a particular valid user. If the attacker impersonates the valid user, they may be able to prevent the user from accessing the server in question.
The second solution is simply difficult to effectively institute – and even when properly done, it does not provide a full solution. It simply makes the attack require more resources on the part of the attacker.