Automate computer using PyAutoGUI

python pyautogui

Hello Techies,

In this article, we are going to discuss how to automate computer using PyAutoGUI.

Doing the same thing over and over again can make you less interested. Automating these types of activities can save your lot of time and help you become more productive in other areas. Then why not try new possibilities which help you doing things using advanced technology.

Many modules can do these things, but in this article, we will use a module called PyAutoGUI to automate GUI and desktop using Python.

PyAutoGUI is an automation module provided by Python to programmatically control keyboard and mouse functions. This module contains almost all the functions that can be performed by keyboard and mouse. We can use these functions to automate mouse and keyboard actions.

Requirements

What you will get to learn

  • Mouse movement and click automaton.
  • Keyboard key press automation.

Projects Setup

To start any python-based project we have to follow common steps of installation. So if you have done all the installation then Cheers!!.

If you have not yet done with this installation then you can refer to my previous blogs. The links are below.

For Windows machine: https://techpluslifestyle.com/technology/how-to-install-django-3-on-windows

For Linux machine: https://techpluslifestyle.com/technology/how-to-install-django-3-on-ubuntu/

Note: No need to install any packages using pip just follow the step till env activation.

Before using the functions of the PyAutoGUI module, you need to install it. To install this module, run the following command in the terminal.

pip install pyautogui

Now PyAutoGUI is ready to use.

Mouse movements and clicks automaton

Now we will see, how to control mouse movements, clicks, drags, and scrolls.

PyAutoGUI size() and position() method

Before automating mouse actions, it is good to know the resolution of the display. We can get the resolution of the display by using the .size() method.

Example: How to get the screen resolution using Pyautogui?

import pyautogui

print(pyautogui.size())

Output

Size(width=1366, height=768)

Now, let’s check how to get the current coordinates of the mouse. You can use the below code.

import pyautogui

print(pyautogui.position())

Output:

Point(x=304, y=619)

By using the position() method of PyAutoGUI, you will get the current position of the cursor.

Now will see how to check if the cursor is on the screen or not. For this, you can use the below code.

Example: How do I find Pyautogui coordinates?

import pyautogui

x, y = pyautogui.position()
print(pyautogui.onScreen(x, y))

Output:

True

Moving the Mouse on Display

So we know some basic methods of PyAutoGUI and the resolution of our display. Now we can move the mouse anywhere on the display within the limits of the display’s resolution. We can move the mouse across the screen using the .moveTo() method of PYAUTOGUI.

Python script to move the mouse every minute:

import pyautogui

pyautogui.moveTo(100,100,duration=1)

We have passed three arguments to the .moveTo() method. The first two arguments are mandatory and indicate the destination location but the third argument is optional. If you want the movement to be immediate, you don’t need to pass the duration as the third argument.

If you want to move the mouse cursor relative to its current position, i.e. you want to move left, right, up, or down from where the mouse is currently, a method to do that is moveRel(), which means move relative.

Example: How do I move the mouse in Pyautogui?

import pyautogui

pyautogui.moveTo(100,500,1)
pyautogui.moveRel(500,200, 1)

Clicking, Scrolling, and Dragging Mouse

PyAutoGUI also provides some more methods like click, scroll, and drag, where you can automate your mouse with these actions.

A mouse click action can be performed using the .click() method. By default, it provides a left click.

Example: How do you click in Pyautogui?

import pyautogui
pyautogui.click() # without arguments
pyautogui.click(100,100,1) # with arguments

It has three arguments passed to it, the x, y coordinates, and the duration.

Here, we are telling our mouse to click on the point (100, 100) after one second. So when your mouse moves to the search bar, which we specified by those coordinates, it will click on it.

The mouse has three options to click; Left, middle, and right mouse buttons. So, we have to specify which button to press.

The left mouse button is usually the default. For example, pyautogui.click(100 ,100, button=’right’). This code right-clicks the point (100, 100).

It also provides a double-click action using the pyautogui.doubleClick() method.

Example: How do you double click in Pyautogui?

import pyautogui
pyautogui.doubleClick()
pyautogui.doubleClick(100,100,1)

PyAutoGUI also provides a scroll action to scroll the mouse up or down. To do this action PyautoGUI provides the .scroll() method.

This method takes an integer as an argument, then scrolls up or down depending on the integer. If you pass a positive integer it will scroll up and for a negative integer it will scroll down.

Example: How to scroll up and down using PyAutoGUI .scroll() method

import pyautogui

pyautogui.scroll(50)
pyautogui.scroll(-50)

The last method for mouse events is dragging, PyAutoGUI provides dragTo() method for this event.

PyAutoGUI’s dragTo() functions have the same parameters as the moveTo() and move() functions. Additionally, they have a button keyword that can be set to ‘Left’, ‘Middle’, and ‘Right’ for the mouse button to hold down while dragging.

Example: How to scroll up and down using PyAutoGUI .scroll() method

import pyautogui
pyautogui.dragTo( 100, 100, 2)

Keyboard key press automation

Pyautogui write() Function

pyautogui.write(‘Python’) is a keyboard method used for typing automation. This function will type the characters in the passed string. To add a delay interval to each letter key press, pass an int or float for the interval keyword argument.

Example: How do you write text in Python?

pyautogui.write('Python world!')                 # prints out "Python world!" instantly
pyautogui.write('Python world!', interval=0.25)  # "Python world!" prints with a quarter second delay after each character

You cannot press the Shift or F1 key, you can only press a single letter key with write().

PyautoGUI press(), keyDown(), and keyUp() Functions

Now let’s check Pyautogui’s keyboard press() method.

To press any keyboard key, call the press() function and pass it a string from pyautogui.KEYBOARD_KEYS such as enter, esc, f1.

Example: How do you press the key in Pyautogui?

pyautogui.press('enter')  # press the Enter key
pyautogui.press('f1')     # press the F1 key

To press multiple keys, just like write(), pass a list of strings to press(). For example:

pyautogui.press(['left', 'left', 'left', 'left'])

Or you can set left presses:

pyautogui.press('left', presses=4)

The following are valid keyboard keys:

['\t', '\n', '\r', ' ', '!', '"', '#', '$', '%', '&', "'", '(',
')', '*', '+', ',', '-', '.', '/', '0', '1', '2', '3', '4', '5', '6', '7',
'8', '9', ':', ';', '<', '=', '>', '?', '@', '[', '\\', ']', '^', '_', '`',
'a', 'b', 'c', 'd', 'e','f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '{', '|', '}', '~',
'accept', 'add', 'alt', 'altleft', 'altright', 'apps', 'backspace',
'browserback', 'browserfavorites', 'browserforward', 'browserhome',
'browserrefresh', 'browsersearch', 'browserstop', 'capslock', 'clear',
'convert', 'ctrl', 'ctrlleft', 'ctrlright', 'decimal', 'del', 'delete',
'divide', 'down', 'end', 'enter', 'esc', 'escape', 'execute', 'f1', 'f10',
'f11', 'f12', 'f13', 'f14', 'f15', 'f16', 'f17', 'f18', 'f19', 'f2', 'f20',
'f21', 'f22', 'f23', 'f24', 'f3', 'f4', 'f5', 'f6', 'f7', 'f8', 'f9',
'final', 'fn', 'hanguel', 'hangul', 'hanja', 'help', 'home', 'insert', 'junja',
'kana', 'kanji', 'launchapp1', 'launchapp2', 'launchmail',
'launchmediaselect', 'left', 'modechange', 'multiply', 'nexttrack',
'nonconvert', 'num0', 'num1', 'num2', 'num3', 'num4', 'num5', 'num6',
'num7', 'num8', 'num9', 'numlock', 'pagedown', 'pageup', 'pause', 'pgdn',
'pgup', 'playpause', 'prevtrack', 'print', 'printscreen', 'prntscrn',
'prtsc', 'prtscr', 'return', 'right', 'scrolllock', 'select', 'separator',
'shift', 'shiftleft', 'shiftright', 'sleep', 'space', 'stop', 'subtract', 'tab',
'up', 'volumedown', 'volumemute', 'volumeup', 'win', 'winleft', 'winright', 'yen',
'command', 'option', 'optionleft', 'optionright']

Let’s check keyUp(), and keyDown() method.

Example:

pyautogui.keyDown('shift')  # hold down the shift key
pyautogui.press('left')     # press the left arrow key
pyautogui.press('left')     # press the left arrow key
pyautogui.keyUp('shift')    # release the shift key

PyAutoGUI hold() function

To make key holding convenient, the hold() function can be used as a context manager and passed a string from pyautogui.KEYBOARD_KEYS such as shift, ctrl, alt, and this key will be held for the duration with the context block.

with pyautogui.hold('shift'):
    pyautogui.press(['left', 'left', 'left', 'left'])

which is equivalent to:

pyautogui.keyDown('shift')  # hold down the shift key
pyautogui.press('left')     # press the left arrow key
pyautogui.press('left')     # press the left arrow key
pyautogui.press('left')     # press the left arrow key
pyautogui.press('left')     # press the left arrow key
pyautogui.keyUp('shift')    # release the shift key

Pyautogui hotkey() Function

Pressing and holding the ‘Alt’ key and then pressing the ‘F4’ key and releasing the ‘Alt’ key after releasing the ‘F4’ key is very complicated. This can only be done using the .hotkey() method. The following example will illustrate the concept.

Example:

pyautogui.hotkey('ctrl', 'shift', 'esc')

Conclusion

The code above demonstrates simple mouse and keyboard automation using the PyAutoGUI library.

You can modify the PyAutoGUI above code as per your requirement and use it to automate your daily repetitive tasks.

I hope you understand the concept of the Python PyAutoGUI . If you have any doubt please comment down below.

Leave a Comment