files

README.md

@@ -1,2 +1,69 @@
-# q-learning-terrain-navigator
+# Q-Learning Terrain Navigator
+This repository demonstrates the application of Q-learning algorithm in a jupyter notebook environment. The Q-learning algorithm is used to navigate a terrain map and learn the optimal path.
 
+## Jupyter Notebook
+<img src="docs/q-table.png" width="600">
+
+
+### Description
+This notebook show how Q-learning works and demonstrate how it can learn to navigate a terrain map and find the optimal path to travel down. This map made using a pygame program that allows you to interactavly draw out the different parts of the terain.
+
+### Features
+- Create a map for the Q-learning algorithm to try using pygame
+- Visualize the map and q-table using a matplotlib
+- Implement a multi-threaded version of Q-learning
+- Compare different hyper paramaters of Q-learning algorithm
+
+### Requirements
+- Python 3.x
+- `jupyter lab` or `jupyter notebook`
+
+```bash
+pip install numpy, matplotlib, threading, tqdm,
+```
+
+
+
+## Pygame Map Builder
+<img src="docs/map_generator.png" width="600">
+
+
+### Description
+This Pygame-based map editor allows users to generate, edit, and save custom maps. Users can define grid sizes, place various terrain types, and automatically apply boundary walls around the grid. The map is saved to a JSON file, and it will load from the file if it exists on startup.
+
+### Features
+- **Adjustable Grid Size**: Use the slider on the right to set the grid size from 5x5 to 15x15.
+- **Terrain Types**: Place different terrain types using keyboard shortcuts.
+- **Boundary Walls**: Automatically creates a boundary with walls (-1000) around the grid.
+- **Save/Load Functionality**: The map is saved to and loaded from `map_data.json`. If you have already created a map it will automatically detect that and load it on start, so you can make simple changes without having to rebuild it again.
+### Controls
+- **Slider**: Adjust grid size (5x5 to 15x15) using the slider on the right panel.
+- **Mouse Click**: Click inside the grid to select a cell and place terrain based on the active key shortcut.
+
+### Terrain Shortcuts:
+- `G`: Place **Goal** (1000)
+- `R`: Place **Road** (-1)
+- `C`: Place **Cliff** (-100)
+- `V`: Place **River** (-10)
+- `M`: Place **Mountain** (-50)
+
+### How to Use
+1. **Start the Program**: Run the Python script to start the map editor.
+2. **Set Grid Size**: Use the slider on the right panel to adjust the grid size between 5x5 and 15x15.
+3. **Edit the Map**: Click on cells and use the keyboard shortcuts to place terrain types.
+4. **Save/Load**: On exiting, the current map is saved to `map_data.json`. The map will automatically load the next time the program is started if the file exists.
+
+### Requirements
+- Python 3.x
+- `pygame` library
+- `numpy` library
+
+### Installation
+Install dependencies using:
+```bash
+pip install pygame numpy
+```
+### Running the Program
+```bash
+./map_generator
+```

map_data.json

@@ -0,0 +1 @@
+[[-1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0], [-1000.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -50.0, -50.0, -1000.0], [-1000.0, -1.0, -100.0, -1.0, -1.0, -1.0, -1.0, -50.0, -50.0, -1.0, -1.0, -1000.0], [-1000.0, -1.0, -100.0, -1.0, -1.0, -1.0, -50.0, -50.0, -100.0, -1.0, -1.0, -1000.0], [-1000.0, -100.0, -100.0, -1.0, -1.0, -50.0, -50.0, -50.0, -100.0, -1.0, -1.0, -1000.0], [-1000.0, -100.0, -1.0, -1.0, -10.0, -10.0, -50.0, -50.0, -1.0, -1.0, -1.0, -1000.0], [-1000.0, -1.0, -1.0, -100.0, -10.0, -10.0, -10.0, -50.0, -1.0, -1.0, -1.0, -1000.0], [-1000.0, -1.0, -100.0, -100.0, -1.0, -10.0, -10.0, -10.0, -50.0, -1.0, -1.0, -1000.0], [-1000.0, -1.0, -100.0, -1.0, -1.0, -1.0, -10.0, -10.0, -10.0, -50.0, -1.0, -1000.0], [-1000.0, -1.0, -100.0, -1.0, -1.0, -1.0, -1.0, -10.0, -10.0, -10.0, -1.0, -1000.0], [-1000.0, 1000.0, -1.0, -1.0, -1.0, -1.0, -1.0, -1.0, -10.0, -10.0, -1.0, -1000.0], [-1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0, -1000.0]]

map_generator

@@ -0,0 +1,162 @@
+#!/usr/bin/env python3
+
+
+import pygame
+import numpy as np
+import json
+import os
+
+# Pygame setup
+pygame.init()
+width, height = 800, 600  # Adjust window size to make space for the shortcut panel
+rows, cols = 10, 10       # Default size of the grid (can be changed)
+cell_size = min(width // (cols + 4), height // rows)  # Adjust cell size based on window and grid size
+
+# Colors
+colors = {
+    'wall': (0, 0, 0),      # Black for wall (boundary)
+    'cliff': (255, 0, 0),    # Red for cliff
+    'road': (128, 128, 128), # Grey for road
+    'goal': (0, 255, 0),     # Green for goal
+    'river': (0, 0, 255),    # Blue for river
+    'mountain': (139, 69, 19), # Brown for mountain
+    'empty': (255, 255, 255) # White for default
+}
+
+# Create initial map array with a dynamic size
+def create_map(rows, cols):
+    # Create a new map array and set boundary values to -1000
+    new_map = np.full((rows, cols), -1.0)  # Default to road (-1)
+    new_map[0, :] = -1000  # Top boundary
+    new_map[-1, :] = -1000  # Bottom boundary
+    new_map[:, 0] = -1000  # Left boundary
+    new_map[:, -1] = -1000  # Right boundary
+    return new_map
+
+# Function to load the map from the JSON file and set the size accordingly
+def load_map():
+    if os.path.exists("map_data.json"):
+        with open("map_data.json", "r") as f:
+            loaded_map = np.array(json.load(f))
+            return loaded_map, loaded_map.shape[0], loaded_map.shape[1]  # Return map and its dimensions
+    return create_map(rows, cols), rows, cols  # If no file exists, return default map
+
+# Load the map and set the initial size based on the file
+map_array, rows, cols = load_map()
+cell_size = min(width // (cols + 4), height // rows)  # Adjust cell size based on loaded grid size
+
+# Create the window
+screen = pygame.display.set_mode((width, height))
+pygame.display.set_caption("Map Editor")
+
+# Slider parameters
+slider_x = width - 100
+slider_y = 350
+slider_height = 200
+slider_pos = slider_y + (slider_height // 2)  # Initial slider position
+min_size = 5
+max_size = 15
+slider_value = rows  # Default slider value corresponds to the loaded grid size
+
+# Function to draw the grid
+def draw_grid():
+    for row in range(rows):
+        for col in range(cols):
+            value = map_array[row, col]
+            if value == -1000:
+                color = colors['wall']  # Black for walls
+            elif value == -100:
+                color = colors['cliff']
+            elif value == -50:
+                color = colors['mountain']
+            elif value == -10:
+                color = colors['river']
+            elif value == -1:
+                color = colors['road']
+            elif value == 1000:
+                color = colors['goal']
+            else:
+                color = colors['empty']
+            pygame.draw.rect(screen, color, (col * cell_size, row * cell_size, cell_size, cell_size))
+            pygame.draw.rect(screen, (0, 0, 0), (col * cell_size, row * cell_size, cell_size, cell_size), 1)
+
+# Function to display shortcut panel and slider
+def draw_side_panel():
+    font = pygame.font.SysFont(None, 24)
+    shortcuts = [
+        "Shortcuts:",
+        "G: Goal (1000)",
+        "R: Road (-1)",
+        "C: Cliff (-100)",
+        "V: River (-10)",
+        "M: Mountain (-50)"
+    ]
+    for i, text in enumerate(shortcuts):
+        img = font.render(text, True, (0, 0, 0))
+        screen.blit(img, (cols * cell_size + 10, i * 30 + 10))
+    
+    # Draw the slider
+    pygame.draw.rect(screen, (150, 150, 150), (slider_x, slider_y, 20, slider_height))  # Slider track
+    pygame.draw.circle(screen, (0, 0, 0), (slider_x + 10, slider_pos), 10)  # Slider knob
+    label = font.render(f"Size: {slider_value}x{slider_value}", True, (0, 0, 0))
+    screen.blit(label, (slider_x - 10, slider_y - 30))  # Display current grid size
+
+# Main loop
+running = True
+dragging_slider = False
+
+while running:
+    screen.fill((255, 255, 255))  # Fill the background
+
+    for event in pygame.event.get():
+        if event.type == pygame.QUIT:
+            # Save map to file on exit
+            with open("map_data.json", "w") as f:
+                json.dump(map_array.tolist(), f)
+            running = False
+
+        # Handle mouse press on slider
+        if event.type == pygame.MOUSEBUTTONDOWN:
+            mouse_x, mouse_y = pygame.mouse.get_pos()
+            if slider_x <= mouse_x <= slider_x + 20 and slider_y <= mouse_y <= slider_y + slider_height:
+                dragging_slider = True
+
+        # Handle mouse release for slider
+        if event.type == pygame.MOUSEBUTTONUP:
+            dragging_slider = False
+
+        # Handle dragging of the slider
+        if dragging_slider:
+            mouse_y = pygame.mouse.get_pos()[1]
+            slider_pos = max(slider_y, min(slider_y + slider_height, mouse_y))
+            # Map the slider position to a value between min_size and max_size
+            slider_value = min_size + (slider_pos - slider_y) * (max_size - min_size) // slider_height
+            rows, cols = slider_value, slider_value
+            map_array = create_map(rows, cols)
+            cell_size = min(width // (cols + 4), height // rows)
+
+        # Handle mouse clicks for grid drawing
+        if not dragging_slider and pygame.mouse.get_pressed()[0]:
+            x, y = pygame.mouse.get_pos()
+            if x < cols * cell_size:  # Only allow clicking inside the grid
+                col, row = x // cell_size, y // cell_size
+
+                # Change the value based on key press
+                keys = pygame.key.get_pressed()
+                if keys[pygame.K_g]:  # 'g' for Goal
+                    map_array[row, col] = 1000
+                elif keys[pygame.K_r]:  # 'r' for Road
+                    map_array[row, col] = -1
+                elif keys[pygame.K_c]:  # 'c' for Cliff
+                    map_array[row, col] = -100
+                elif keys[pygame.K_v]:  # 'v' for River
+                    map_array[row, col] = -10
+                elif keys[pygame.K_m]:  # 'm' for Mountain
+                    map_array[row, col] = -50
+
+    # Redraw grid and side panel
+    draw_grid()
+    draw_side_panel()
+    pygame.display.update()
+
+pygame.quit()