wget files matching regex 1

wget files matching regex

ByAnurag batra 
# Getting http://some/site/*my_pattern*
SITE="http://some/site/"
PATTERN="my_pattern"
wget -nv -O- $SITE | 
  grep $PATTERN | 
    awk -v pat=$PATTERN -F '"' '{ for (i=1; i<=NF; ++i) { if ($i ~ pat) print $i } }' |
      while read x; do 
      	if [[ ! $x = ">" ]] ; then
      	  wget $SITE$x &>/dev/null
      	  echo $x
      	fi

Here is what the above code is Doing:
1. Get the site’s html
2. Filter out the lines that contain the pattern
3. For each line, print the part of the line that contains the pattern
4. For each line, if it’s not a ‘>’ (which is the html for the next page), then download the file and print the filename

Similar Posts

existing repository

existing repository

ByAnurag batra

git remote add origin https://github.com/Polsa96/githubBasicAlgorithms.git git branch -M main git push -u origin main Here is what the above code is Doing: 1. git init – Initializes a git repository in the current directory 2. git add . – Adds all files in the current directory to the staging area 3. git commit -m “Initial…

android studio kvm is required to run this avd

android studio kvm is required to run this avd

ByAnurag batra

#Starting with Ubuntu 18.04 and Linux Mint Tara you need to install qemu-kvm sudo apt install qemu-kvm #Check the ownership of /dev/kvm ls -al /dev/kvm #Check which users are in the kvm group grep kvm /etc/group #Output from the above command #kvm:x:some_number: #If there is nothing rightwards of the final :, there are no users…

ModuleNotFoundError: No module named ‘django.db.migrations.migration’

ModuleNotFoundError: No module named ‘django.db.migrations.migration’

ByAnurag batra

# Django is probably corrupted; reinstall it pip3 uninstall Django pip3 install Django Here is what the above code is Doing: 1. It’s importing the Django module. 2. It’s creating a new Django project named mysite. 3. It’s creating a new Django app named polls. 4. It’s running the development server.