How to create simple but effective log reports using basic Linux commands

Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInShare on VKPin on Pinterest

Lately I’ve noticed an increased number of failed SSH attempts on my server. After inspecting the auth.log file, I saw a lot of repeated ssh attempts for root user:

$ tail -f /var/log/auth.log
Jul 27 09:43:20 node1 sshd[8810]: Failed password for root from 182.100.67.119 port 1999 ssh2
Jul 27 09:43:20 node1 sshd[8808]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=61.177.172.22  user=root
Jul 27 09:43:20 node1 sshd[8810]: Received disconnect from 221.194.44.212 port 45339:11:  [preauth]
Jul 27 09:43:20 node1 sshd[8810]: Disconnected from 221.194.44.212 port 45339 [preauth]
Jul 27 09:43:20 node1 sshd[8810]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=221.194.44.212  user=root
Jul 27 09:43:21 node1 sshd[8803]: Failed password for root from 182.100.67.119 port 1999 ssh2
Jul 27 09:43:28 node1 sshd[8812]: message repeated 4 times: [ Failed password for root from 182.100.67.119 port 1999 ssh2]
Jul 27 09:43:21 node1 sshd[8803]: error: maximum authentication attempts exceeded for root from 182.100.67.119 port 1999 ssh2 [preauth]
Jul 27 09:43:21 node1 sshd[8803]: PAM service(sshd) ignoring max retries; 6 > 3
Jul 27 09:43:22 node1 sshd[8812]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=221.194.44.212  user=root
Jul 27 09:43:24 node1 sshd[8812]: Failed password for root from 221.194.44.212 port 54140 ssh2
Jul 27 09:43:28 node1 sshd[8812]: message repeated 2 times: [ Failed password for root from 221.194.44.212 port 54140 ssh2]

That’s a lot of information to go through, and sometimes I just want to see the things I care about. There are lots of tools out there that can parse the log files that can generate nice reports for you, but sometimes you might not have the luxury of installing new stuff, or just want to save resources or you have limited rights for installing packages on that server. So what to do in this case?

Fortunately, by leveraging the power of simple Linux commands, I can see the relevant information that I am looking for. For example, I just want to see all the IP addresses from which the requests are coming from. Let’s start simple.

First, we’ll need to know how many entries we’ve had so far:

$ wc -l /var/log/auth.log
112769 /var/log/auth.log

Let’s see how many of these errors were due to repeated failed ssh attempts for root:

$ grep "message repeated .* Failed password for root" /var/log/auth.log | wc -l
8226

That’s better. Now, let’s generate a list of the IP addresses from which the requests came from. For this we’re going to use the awk command. Looking at the lines, the 16th column represents the IP address. So let’s print it:

$ grep "message repeated .* Failed password for root" /var/log/auth.log | awk '{print $16}'
..............
182.100.67.119
182.100.67.119
182.100.67.119
121.18.238.125
121.18.238.125
182.100.67.119
61.177.172.22
182.100.67.119
182.100.67.119
182.100.67.119
182.100.67.119
121.18.238.125
121.18.238.125

Now, that’s what we want, but hang on a second! The same IP addresses appears several times. No stress, we can just select the unique IP addresses, and also sort them while we’re at it:

$ grep "message repeated .* Failed password for root" /var/log/auth.log | awk '{print $16}' | sort | uniq
.............
210.51.191.26
221.194.44.212
221.194.47.224
95.141.101.32
61.177.172.22
121.18.238.125
182.100.67.119

That’s much better. Now, we have a list of all the unique IP addresses that tried to brute force the server. Typically, these are hordes of botnets trying to find targets just by trying every IP address in certain subnets and trying to brute-force login by using dictionary attacks on well-known accounts like “root”.

Having obtained this list of IP addresses, we can start digging and trace the IP addresses back to their location, but this is out of the scope of this post (maybe the next one).

In order to avoid being a victim of these attackers, it’s always a good idea to:

  • Use a tool like fail2ban to ban for 24 hours requests coming from these IP addresses
  • Disable logging in as root in SSH
  • If possible, use public-key authentication and completely disable password-based authentication
  • Use SSH on a different port than the default one (for example, change 22 with 2222)

Conclusion

We saw how easy it is to generate simple but efficient reports, just by using extremely basic UNIX/Linux commands. Okay, this one was extremely basic, but it got me the information that I needed. If you ever need more information, you can always print the columns that you are interested in and perform further manipulation on that data.

126 thoughts on “How to create simple but effective log reports using basic Linux commands

  1. I as well as my friends appeared to be viewing the nice thoughts located on your site while all of the sudden I got a horrible feeling I never expressed respect to you for those techniques. The women had been happy to read through all of them and already have in fact been enjoying those things. I appreciate you for actually being quite thoughtful and then for going for variety of useful ideas most people are really wanting to be informed on. My personal sincere regret for not expressing appreciation to you sooner.

    http://www.zvodretiluret.com/

  2. Unquestionably believe that which you stated. Your favorite justification seemed to be on the net the easiest thing to be aware of. I say to you, I certainly get irked while people consider worries that they just don’t know about. You managed to hit the nail upon the top as well as defined out the whole thing without having side effect , people could take a signal. Will probably be back to get more. Thanks

    http://www.vividleds.us/

  3. Purely to follow up on the up-date of this matter on your web page and want to let you know simply how much I appreciated the time you took to generate this beneficial post. Inside the post, you actually spoke regarding how to definitely handle this matter with all convenience. It would be my pleasure to get some more suggestions from your website and come up to offer other folks what I have benefited from you. Thank you for your usual terrific effort.

    http://commissionoverdrive.online

  4. Jane Iredale 熣燦滋潤豐唇蜜(Red) Jane Iredale 熣燦滋潤豐唇蜜(Red)Jane Iredale天然成份,創新技術,不用鏡子,輕快地一塗,屬於您的健康紅潤色彩漸漸出現,即時趕走灰暗倉白。更可當作胭脂使用,在胭脂位置塗上一點點後,用手指即時輕輕推勻,青春粉紅色彩即時散發。 營養價值:以葡萄柚、橙、檸檬皮粹取的全天然成分配合triisostearyl citrate 的皮膚柔潤劑來激發皮膚上的酸性值而產生個人的粉

    https://cosmetic.wiki/tags/最高防曬系數-泰國

  5. Importante empresa, solicita TELEMERCADERISTA, con experiencia minima un año en telemercadeo; estudiante de carreras administrativas comerciales, Tecnico on Tecnologo en Administracion, Comercio, Mercadeo Afines. Importante empresa del sector Salud requiere Técnico en carreras Administrativas y / Contable con experiencia mínima de 1 año en solicitud y trámite de autorizaciones de servicios hospitalarios de los pacientes, diligenciar e informar a los usuarios los valores a cancelar y realizar censo de asignación de camas. Se requieren Ingeniero de Sistemas con experiencia certificada en desarrollo de software de más de 4 años.

    http://dmitriykonkm.webteksites.com

  6. Vi jugar a Pirri, Breitner, Camacho en sus inicios, a Netzer, todavía tengo recuerdos de un Madrid Velencia en el que jugaba Kempes y en el que el Madrid le dio la vuelta a una liga igualadísima muy parecida a la que ha ganado este año, me hicieron Socio y a partir de ese momento no faltaba un Domingo a para ver al Madrid. Las comodidades incluyen frigorífico y sofá cama, además de un servicio de limpieza disponible todos los días.

    http://damon0611ag.electrico.me

  7. Teniendo en cuenta que en estos avisos solo contamos los realizados de la marca Aspes, incluyendo calderas, calentadores u otros tipos de electrodomésticos de esta fantastica marca. Y es que aunque no seamos el servicio técnico oficial de Aspes en Madrid, nos tomamos nuestra profesión con igual seriedad y profesionalidad para ofrecer el servicio igual que el oficial.

    http://antonuvarokr.intelelectrical.com

  8. Pensamos que la estrechez no está en el Acuerdo General de la Habana, sino en la visión que el gobierno tiene sobre las causas y consecuencias de la confrontación. Hoy 18 de octubre 2012 en Oslo (Noruega) y como jefe único del equipo negociador oficial del gobierno colombiano para la segunda fase del acuerdo pactado en la Habana, quedó claro nuevamente su papel de demoledor de acuerdos” que le señala Álvaro Leiva. LA HABANA / ANNCOL / OCTUBRE 19 / Con la instalación de la mesa de diálogo en las afueras de Oslo, comenzó el proceso de paz que desarrollarán las FARC-EP y el gobierno de Colombia. Pero pienso que ahora hay tareas más importantes para hacer en Colombia y en el mundo también.

    http://chad6804pd.tubablogs.com

  9. Para empresa de mecanizado ,se necesita fresador CNC con conocimientos de CAM y con experiencia en manejo de centros de mecanizado. Si quieres formarte para trabajar en un sector con demanda de profesionales, Vitae Aeronáutica S.L. te da la oportunidad de aprender el oficio de operario para las empresas de mecanizado.

    http://lasvegasseocompani2dq.realscienceblogs.com

  10. MIPESA GRUPO EMPRESARIAL, engloba diversas industrias auxiliares de mecanizado, poniendo a su disposición desde nuestros talleres, servicios de fabricación integrales, como mecanizado por arranque de viruta, corte láser, punzonado, plegado, soldadura y montaje de maquinaria y conjuntos. Son innumerables los negocios que mantienen su confianza en nosotros para la producción de piezas en Galicia de características muy diversas, puesto que tienenen nosotros a un claro referente por precio y calidad. Mecanizados Garrigues cuenta con una gama de maquinarias adaptadas a las nuevas tecnologías para conseguir una precisión y calidad que queda reflejada en todos los trabajos que realizamos.

    https://industria.pbworks.com

  11. Fabricamos piezas unitarias y también series de todos los tamaños y en gran variedad de materiales. Utilizamos las herramientas más eficaces e innovadoras del mercado, y te ofrecemos resultados de primera calidad que no encontrarás en ningún otro lugar. Sin la primera máquina-herramienta de mecanizado, la mandrinadora de Wilkinson, no se hubiera podido fabricar la máquina de vapor que posibilitó la Revolución Industrial.

    http://lasvegasseocompani2dq.realscienceblogs.com

  12. JOE BERNSTEIN AT THE ETIHAD: With City already through to the Champions League knockout stages – Toure was drafted in against Feyenoord to see if there was still life in the old dog yet. Yaya Toure dictates tempo for Manchester City against Feyenoord as Ivorian is handed surprise start to see if there was still life in the old dog

    https://bit.ly/2JCArSc

  13. Garry Monk has described Arsene Wenger’s call for him to be crowned manager of the year as a ‘massive compliment’. But the Swansea boss still wants to beat Arsenal on Monday night. Swansea boss Garry Monk flattered by Arsene Wenger praise before Arsenal clash

    https://bit.ly/2S3sXus

Leave a Reply

Your email address will not be published. Required fields are marked *