Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - noman

Pages: [1] 2
1
Hardware / IT-Monkey Lab Hardware - end of 2017
« on: November 25, 2017, 11:54:37 AM »
I decided to create this topic about what hardware we use in our test lab her at IT-Monkey, after quite a few comments from people we have been in contact whit that they have no access to brand new hardware and that hardware is so expensive that they can not afford systems and lab equipment like what we have here. This is why I decided to share this list of what our lab hardware is and what our servers are made out of, and to showcase that you do not need the lates and greatest in a home-lab or test-lab at least not when you are starting out and want to learn as it was the goal for my home-lab when I first started out in 2010.


Physical Servers:
At this point in time we at IT-Monkey have 3 different DIY server in our lab-rack, these are:

1. Storage server, running FreeNAS 9.10.2-U6 This system was purpose build as a central storage server for my home office, home-lab and business, it is the only system that I have bought everything brand new from the retailer that sits in my server-rack.
2. Lab1-Server, hypervisor running VMWare ESXi 5.5
3. Lab2-Server, currently have no storage and OS, going to be used as a hypervisor standalone or cluster depending on software and pricing..
these are just generic names and not the actual names of the servers, for security reasons,

FreeNAS-Server Hardware:
Code: [Select]
Chenbro RM23212 2U rackmount whit rails, 12-hotswap hdd bays
Motherboard: Supermicro X11SSL-F (MicroATX, Socket 1151. C232 Chipset and DDR4)
Processor: Intel Xenon E3-1245v5 4 Cores at 3.5Ghz
Memory: Crucial DDR4 ECC unreg 64Gb 2133Mhz KIT
Powersupply: SeaSonic 400w 80-Plus Bronze
CPU Cooler: Noctua NH-L9x56
Raid Controller: LSI MegaRAID 9211-8 (Flashed to IT 20 Firmware)
NIC: Intel PRO 1000 PT (EXPI9404PT) 4-port
OS Drive: SanDisk Cruszer 32Gb
HDD 01: Seagate Ironwolf ST4000VN008 64MB 4TB
HDD 02: Seagate Ironwolf ST4000VN008 64MB 4TB
HDD 03: Seagate Ironwolf ST4000VN008 64MB 4TB
HDD 04: Seagate Ironwolf ST4000VN008 64MB 4TB
HDD 05: Seagate Ironwolf ST4000VN008 64MB 4TB
HDD 06: Seagate Ironwolf ST4000VN008 64MB 4TB
HDD 07: Seagate Ironwolf ST4000VN008 64MB 4TB
HDD 08: Seagate Ironwolf ST4000VN008 64MB 4TB
HDD 09: Western Digital Red WD40EFRX 64MB 4TB
HDD 10: Western Digital Red WD40EFRX 64MB 4TB
HDD 11: Western Digital Red WD40EFRX 64MB 4TB
HDD 12: Western Digital Red WD40EFRX 64MB 4TB
Fan 1: GELID Silent PRO 8 PWM
Fan 2: GELID Silent PRO 8 PWM
Fan 3: GELID Silent PRO 8 PWM

Note; This system was purpose build as a central storage server for my home office, home-lab and business, it is the only system that I have bought everything brand new from the retailer that sits in my server-rack.

Lab1-Server Hardware:
Code: [Select]
Case: X-Case X255F 2U rackmount whit rails
Motherboard: MSI Z77A-G45 LGA1155
Processor: Intel i7-3770 4-cors 3.40 GHz
Memory: Corsair Vengeance DDR3 1600mhz 32Gb KIT
Powersupply: Corsair CX500M, 500w 80-Plus Bronze
CPU Cooler: Noctua NH-L9x56
Raid Controller: LSI MegaRAID 9211-8 (Flashed to IT 20 Firmware)
NIC: Intel PRO 1000 PT (EXPI9404PT) 4-port
OS Drive: SanDisk Cruszer Fit 8Gb
Internal HDD 1: Intel 335 240Gb
Internal HDD 2: Intel 335 240Gb
Internal HDD 3: Western Digital Red WD30EFRX 64MB 3TB
Internal HDD 4: --
Fan 1: GELID Silent PRO 8 PWM
Fan 2: GELID Silent PRO 8 PWM
Fan 3: GELID Silent PRO 8 PWM

Note; Currently running VMWare ESXi 5.5 and a Windows 2012r2 DC network, thinking about changing hypervisor to something else

Lab2-Server Hardware:
Code: [Select]
Case: X-Case X255F 2U rackmount whit rails
Motherboard: Gigabyte GA-Z68XP-UD3 LGA1155
Processor: Intel i7-3770 4-cors 3.40 GHz
Memory 1: Corsair Vengeance DDR3 1600mhz 16Gb KIT
Memory 2: Corsair Vengeance DDR3 1600mhz 8Gb KIT
Powersupply: Andersson Mission SG GGP 550w 80-Plus
CPU Cooler: Noctua NH-L9x56
Raid Controller: LSI MegaRAID 9211-8 (Flashed to IT 20 Firmware)
NIC: Intel PRO 1000 PT (EXPI9404PT) 4-port
OS Drive: SanDisk Cruszer Fit 8Gb
Internal HDD 1: --
Internal HDD 2: --
Internal HDD 3: --
Internal HDD 4: --
Fan 1: GELID Silent PRO 8 PWM
Fan 2: GELID Silent PRO 8 PWM
Fan 3: GELID Silent PRO 8 PWM

Note; Server has no storage, server is currently being used as a bench testing server, has support for mSATA SSD, thinking about a hypervisor cluster, not sure what OS to use

As you can see both Lab1 and Lab2 server is built out of old consumer grade hardware and have prior to us getting them been used as gaming machines, where we simply refurbished the motherboard and CPU's moved it to a rack-mount case and added appropriate sized coolers and some other spare parts that we had laying around and most of those are not needed to run a basic test-lab.

Network:
As for our network we do not have very much hardware at all and you really do not need much besides a good Managed Switch whit enough ports to connect it together

For our lab we currently have the following networking equipment:
1. ISP Modem
2. SG-4860 pfSense Firewall
3. Cisco SB SG200-26p Switch

As you know there is no internet connectivity whiteout the ISP and their Modem, then there is the pfSense SG-4860 appliance we use this to separate home network, lab network and office network, then it is the Cisco Switch not much to it plug it in and connect devices to it.

As I am writing this post I am about to redo some of my lab setup so if there is intrest for it I can make a post about setting up your own home-lab from scratch whit more detailed information on how everything is interconnected whit each other

2
Off-Topic Therapy / Re: wallhack cs go pobierz
« on: June 29, 2017, 08:55:34 AM »
We are not against talking about games here, but please use english

3
Off-Topic Therapy / Re: minecraft do pobrania
« on: June 29, 2017, 07:17:47 AM »
It is okay to discuss minecraft, games are a part of IT, but please follow the one single rule of using English on this forum

4
Off-Topic Therapy / Re: Porn
« on: June 29, 2017, 07:13:13 AM »
If this was a porn site I would have done a better job keeping track as it would make me more money than this site does...

I do not want porn on this site, as this is for IT related stuff, unless the discussion is technical on how to setup a porn website or cdn I am not interested... please report all and any bad content you find and I will try cleaning up as much that I can

5
Looks good so far, but we should add a note that zabbix client or snmp needs to be installed \ configured on the server and devices before the discovery rules are enabled, otherwise it will only be detected by the Ping rule, and the monitoring will be does it respond to ping every x seconds (I added the Ping rule as a catch all other devices so that you know what devices that need some extra attention)

Also I would recommend to disable the Admin user and create your own user for login even tho you have changed the password and you trace the logs and it says admin did the change, not Mello or Noman in the logs, since this is a enterprise level software it should have a note about that.

We should also add in setting up the email and sms alerting before creating the discovery rules, whit is the point of having a monitoring system without getting alerted when it is triggered ?

This is just some of the thougts I have from laying sick in beed whit my laptop

6
Navigate to Configuration -> Action -> Event source: Discovery -> Create Action -> Operation, see the fb post for screenshot mr. Hill asked about the same yesterday

7
These are good instructions, but you are missing a whole lot of Zabbix configurations in between setting a new system password and accessing the webgui, I have alot of this wirtten up in a document here as generic notes, but ost of that information is also found in the documentation: https://www.zabbix.com/documentation/3.2/manual/appliance

This is what I have so far for zabbix installation, there are missing a few pices and nmap and sudo access but that is easy added on a later stage

Zabbix Installation
Download ISO file from: https://sourceforge.net/projects/zabbix/
Latest version in time of writing is: 3.2.6
Latest version is based on a customized Ubuntu 14.04 release
Install server whit mysql option (would love a nginx and mariadb version), follow the onscreen instructions
Systems default credentials are:
Core Server System:
User: appliance  \ Password: zabbix
Database:
User: root  \ Password: <random>
User: zabbix  \ Password: <random>
Database passwords are randomly generated during the installation process.
Root password is stored to /root/.my.cnf file, it is not required to input a password under the “root” account.
To change the database user password it has to be changed in the following locations:
MySQL;
/etc/zabbix/zabbix_server.conf;
/etc/zabbix/web/zabbix.conf.php.
Zabbix frontend WebGUI
User: Admin  \ Password: zabbix

Login to system shell and get root access and change the root password; User: appliance  \ Password: zabbix
sudo –I
this changes your user appliance into the root user
passwd
this lets you change the password of the current user, now you have gotten your root password set to something known, I recommend setting a strong password as root user have full access to your server and it should not be trivial to get a hold of
I like to work from a SSH connection so edit /etc/ssh_config and /etc/sshd_config to allow root to access the system from a SSH tunnel at least until you are done making changes to the core system
In /etc/ssh_config I have the default settings except
# PasswordAuthentication yes  is changed to PasswordAuthentication yes
added the line: PermitRootLogin yes
Save and exit
In /etc/sshd_config I have the default settings except
# Authentication:
LoginGraceTime 360
PermitRootLogin yes
StrictModes yes
Save and exit
Now restart SSH server whit command: service ssh restart
when this is done open your favorite terminal or console and connect to your server I like to use Putty  - http://www.putty.org/
Now if you are like me and are running Zabbix Server as a VM the next thing you would like to do is install the VM drivers and toolkits so that you get better performance of your monitoring VM
For VM’s running on a vmware platform the commands are as following:
apt-get update  or sudo apt-get update  if you are not at root level
apt-get install open-vm-tools  or sudo apt-get install open-vm-tools  if you are not at root level
reboot or sudo reboot if you are not at root level
For VM’s running on VirtualBox platform the commands are as following:
apt-get update  or sudo apt-get update  if you are not at root level
apt-get install build-essential module-assistant or sudo apt-get install build-essential module-assistant if you are not at root level
m-a prepare or sudo m-a prepare if you are not at root level

Now click "Devices > Insert guest additions CD image" in the virtualbox window. This will insert the guest additions cd image into the guest OS
The cd-image should be located on the path /media/root/VBOXADDITIONS_X.X  or /media/appliance/VBOXADDITIONS_X.X   depending on what user level you are at
Navigate to the directory and run the script named VBoxLinuxAdditions.run
cd /media/root/VBOXADDITIONS_X.X or cd /media/appliance/VBOXADDITIONS_X.X
./VBoxLinuxAdditions.run or sudo ./VBoxLinuxAdditions.run if you are not at root level
When it is done you should see the following line:
Building the VirtualBox Guest Additions kernel modules ...done.
reboot or sudo reboot if you are not at root level
After the server has rebooted we want to change some of the system settings to match our environment and location, this includes timesync, zoneinfo, dns
Check that you have the correct nameservers and dns server specified for your server so that zabbix is able to resolve IP’s into hostnames this is done by editing  /etc/resolv.conf

nameserver 10.99.99.1
nameserver 192.168.55.254
search vmlab.lan
search qnett.lan
This tells the Zabbix server that my dns server is at IP 10.99.99.1 and that my domain is vmlab.lan, if you have a router whit dhcp your server should have gotten this information automaticly, if you would like to add a custom server you just enter each value as its own line.
Now to change the zoneinfo and localization of the server this is done by editing the /etc/localtime file this is done by copying the right zone file over
The zone files are located under: /usr/share/zoneinfo/
/usr/share/zoneinfo /Africa
/usr/share/zoneinfo /America
/usr/share/zoneinfo /Asia
/usr/share/zoneinfo /Europe
In my case I would have to set it to /usr/share/zoneinfo /Europe/Oslo as I am in Norway
cp /usr/share/zoneinfo/Europe/Oslo  /etc/localtime
You would need to select the city closest to your location

Now we need to change the default Zabbix timezone as it it hardcoded to be Europe/Riga this is done by editing the file /etc/apache2/conf-available/zabbix.conf
As per default it will try to synchronize its clock whit the Ubuntu servers, but we would like to synchronize it whit our local time servers so all the systems in our data-center keep the same time, as this is very important for monitoring, logs and reporting purposes we need to install NTP services as this is not installed whit the zabbix installer iso
sudo apt-get update
sudo apt-get install ntp
If you have followed some of my other guides you might recall that I configured my pfSense firewall to act as a local time server for my local network, and I want the zabbix server to get its time synced from that pfSense firewall like any other device
To do this we need to edit the file /etc/ntp.conf
# Specify one or more NTP servers.
server 10.99.99.1 prefer iburst
server pfsense.vmlab.lan prefer iburst
server 0.no.pool.ntp.org
server 1.no.pool.ntp.org
server 2.no.pool.ntp.org
server 3.no.pool.ntp.org
As you can see I have set it to prefer my local pfSense firewall whit both IP and DNS and then added the Norwegian servers of the ntp.org project this is incase my ntp server running at the firewall could have issues one day and it would then fall back to use close external timeservers so it does not lose its time configuration
Now restart NTP whit command: sudo service ntp restart
Test that NTP is working correctly:  watch ntpq -cpe –cas
More information about ntp on Ubuntu servers are here: https://blogging.dragon.org.uk/setting-up-ntp-on-ubuntu-14-04/
When the server is rebooted navigate to its webgui and login whit Admin:zabbix
We want to configure 3 things on the server, which is Host Group, Discovery rules and Action rules
Host Group: This is just a sorting group and serves no other purpose than labeling and sorting different type of devices and servers so you can apply different monitoring rules and parameters to different Host Groups
Discovery Rules: These are the rules that you create that define how and where the Zabbix Server should look for network devices and servers, in short here is where you define the rules for SNMP, PING or Zabbix Client.
Action Rules: These are the rules that says what should happen whit newly discovered devices and servers, should they be added to a Host Group, and should they be assigned a monitoring pattern based of what type of device that were detected whether it is a router or switch using SNMP or a Windows Domain Controller responding to PING or Zabbix Client installed.
This is the very basic configuration needed to get the monitoring going on Zabbix
Navigate to Configuration -> Host Groups -> Create new Host Group
I create the following 3 Host Groups
myNetwork.Zabbx
myNetwork.SNMP
myNetwork.PING
Navigate to Configuration -> Discovery -> Create Discovery rule
I create the following 3 Discovery rules
Discovery.Zabbix
Discovery.SNMP
Discovery.PING
Discovery.Zabbix
Name: Discovery.Zabbix
Discovery by proxy: No Proxy
IP range: 10.99.99.0-254
Delay (in sec): 3600
Checks:  Zabbix agent – default values, key= system.uname
Device uniqueness criteria: IP address
Enabled: NO (unchecked)

Discovery.SNMP
Name: Discovery.SNMP
Discovery by proxy: No Proxy
IP range: 10.99.99.0-254
Delay (in sec): 3600
Checks:  SNMPv1 agent and SNMPv2 agent – default values
Device uniqueness criteria: IP address
Enabled: NO (unchecked)
Discovery.PING
Name: Discovery.PING
Discovery by proxy: No Proxy
IP range: 10.99.99.0-254
Delay (in sec): 3600
Checks:  ICMP Ping
Device uniqueness criteria: IP address
Enabled: NO (unchecked)
Navigate to Configuration -> Action -> Event source: Discovery -> Create Action
Here I create 3 new Action rules
Auto.Discovery.Zabbix
Auto.Discovery.SNMP
Auto.Discovery.PING
Auto.Discovery.Zabbix
Action:
Name: Auto.Discovery.Zabbix
Type of calculation: And/Or
Conditions:
A   Discovery check = Discovery.Zabbix: Zabbix agent "system.uname”
B   Host IP = 10.99.99.0-254
Enable: Yes (checked)
Operation:
1. Send message about new device
2. Operation details:
Add to host groups: myNetwork.Zabbx
Link to templates: Template App Zabbix Agent
Enable host
Set host inventory mode: Automatic
Auto.Discovery.SNMP
Action:
Name: Auto.Discovery.SNMP
Type of calculation: And/Or
Conditions:
A   Discovery check = Discovery.SNMP: SNMPv1 agent "zabbix"
B   Discovery check = Discovery.SNMP: SNMPv2 agent "zabbix"
C   Host IP = 10.99.99.0-254
Enable: Yes (checked)
Operation:
1. Send message about new device
2. Operation details:
Add to host groups: myNetwork.SNMP
Link to templates: Template SNMP Device, Template SNMP Disks, Template SNMP Generic, Template SNMP Interfaces and Template SNMP Processors
Enable host
Set host inventory mode: Automatic
Auto.Discovery.PING
Action:
Name: Auto.Discovery.PING
Type of calculation: And/Or
Conditions:
A   Discovery check = Discovery.PING: ICMP ping
B   Host IP = 10.99.99.0-254
Enable: Yes (checked)
Operation:
1. Send message about new device
2. Operation details:
Add to host groups: myNetwork.PING
Link to templates: Template ICMP Ping
Enable host
Set host inventory mode: Automatic
Navigate to Configuration -> Discovery
Now you can enable the following rules
Discovery.Zabbix
Discovery.SNMP
Discovery.PING

When these rules are enabled let it run for a couple of hours as scanning the network for all your devices will take some time depending on your network layout, how many devices that are on it and the hardware of your Zabbix Server


8
Networking / Zabbix Appliance 3.2.6 inconsistencies
« on: June 08, 2017, 10:09:30 AM »
Hello everyone, I just did a fresh lab deployment of the Zabbix Appliance 3.2.6 and I noticed some inconsistencies from my old lab deployment of Zabbix Appliance 3.0.4

Both ISO files where download from: https://sourceforge.net/projects/zabbix/

Hostname on 3.0.4 is set to: zabbix
Hostname on 3.2.6 is set to: ubuntu

This should be set to zabbix or zabbix-appliance on all versions or a even better option would be a prompt during install to set the hostname, also if you would use Static IP or DHCP configuration, this is a enterprise level server appliance to be deployed in lots of different network environments after all.

Another issue is whit the scrip: Detect operating system,
Command: sudo /usr/bin/nmap -O {HOST.CONN} 2>&1

Does not work as it is not added into sudoers list and nmap is not not installed so after editing it to allow to run the sudo command, it gives error command not find as nmap is not installed to begin whit, this should have been in there to begin whit or this option should have been removed from the front end

This issue is on both version, lucely these are minor issues that are easy to fix, just add the zabbix usergroup to sudo users whit visudo, and then install nmap on the device


Running Script on a detected host, no changes made


Editing sudo users to allow zabbix users to run sudo commands


Running OS Detection script against detected host after sudo rights


Running OS Detection script after nmap was installed

9
Configure pfSense as HTTPS \ SSL Proxy filter using Squid and SquidGuard!

This is a short write-up of how I got pfSense 2.3 and 2.4-Beta to act as an Proxy filter for ssl and https traffic without the needs of installing or configuring any client side settings or certificates, all configurations are done on the pfSense Firewall itself.
Tools needed:

Web-browser
Putty or similar console emulator
Notepad or Notepad++
WinSCP (Optional) gives you graphical text editor over ssh, good for beginners

All the steps below can be done directly on the firewall using only the GUI or SSH connections but for beginners it would be easier to use tools like Notepad++ and WinSCP to edit the configuration files needed for this to work

Step 1. Configuring the root Certificate Authority (rootCA)
This is probably the part that is most confusing for people and why their setups have failed, Squid need to have a CA assigned to it so that it is able to decrypt parts of the HTTPS header so that it can determine what to do whit that traffic, otherwise all traffic is passed.

I used the built-in openssl tool of pfSense to generate this rootCA, for this you need to ssh in to your firewall or connect to it over console, when at the console menu select option 8 Shell, when you are at the shell prompt you need to manually edit the openssl to give you the necessary prompts and questions for you to configure the rootCA

vi /etc/ssl/openssl.cnf

under the [ REQ ] option change the following line from: prompt=no to prompt=yes
under the [ V3_REQ ] option change the following line from: basicConstraints=CA:FALSE to basicConstraints=CA:TRUE
Then save and quit (to save and quit vi editor use :wq!).
Now we would make a known location in the filesystem to save our rootCA and key file as they need to be imported into pfSense GUI at a later stage, I like to use /tmp for any temporary files

Command:
mkdir /tmp/Proxyfiles

Now move to the folder you created whit:
cd /tmp/Proxyfiles

When you’re in this folder you are ready to start the openssl tool and start to create your rootCA, you start whit generating your KEY file by running the command:

openssl genrsa -out myProxykey.key 2048

This will create an rsa key file named myProxykey.key that we use to sign our rootCA whit in the next command for generating the pem file for the rootCA
Create a pem file signed whit key using command:

openssl req -x509 -new -nodes -key myProxykey.key -sha256 -days 365 -out myProxyca.pem

This will prompt you to answer some questions to generate the needed pem file in my case it is as below, you need to change this for where you are

US []:NO    - Country code
Somewhere []:Oslo  - State or province
Somecity []:Oslo   - Your city or town
CompanyName []:IT-Monkey   - Name of your company or business, "make something up if you’re a home user"
Organizational Unit Name (eg, section) []:IT-Department  - What part of the company issued the cert, can also be left blank
Common Name (eg, YOUR name) []:Admin    - Your name or identity in the company
Email Address []:admin@it-monkey.local  - Your contact email

At this point you should have 2 files in your /tmp/Proxyfiles directory

myProxyca.pem
myProxykey.key

This can be double checked whit command:

ls -la

If both are there then you are ready to download them and exit the shell environment and continue to the GUI of pfSense, if not you need to look over any error or try again.

To download these files I like to use WinSCP as its fast to navigate and find the files you want to move you can also do this from pfSense Diagnostic -> Command Prompt option and select the file path of:

/tmp/Proxyfiles/myProxyca.pem
/tmp/Proxyfiles/myProxykey.key



In the download box, this is somewhat slower to navigate but works just fine.

Now as you have these files on your desktop or computer you need to open them in a text editor, I prefer Notepad++ as it’s able to adjust the formatting layout of the text but any text editor will work as you will need to copy paste the information in these files to fields in the pfSense GUI.
Navigate to System -> Cert. Manager -> CA's
Here you want to add a new CA

Descriptive name: SquidCA
Method: Import an existing Certificate Authority
Certificate data: Copy \ Paste the info from myProxyca.pem file
Certificate Private Key: Copy \ Paste the info from myProxykey.key file
Save and apply




now you should see your SquidCA (rootCA) populated under System -> Cert. Manager -> CA's whit all the info you provided in the shell prompt and you are done whit Step 1.

You might want to undo the changes from the /etc/ssl/openssl.cnf file before proceeding

vi /etc/ssl/openssl.cnf

under the [ REQ ] option change the following line from: prompt=yes to prompt=no
under the [ V3_REQ ] option change the following line from: basicConstraints=CA:TRUE to basicConstraints=CA:FALSE

Then save and quit (to save and quit vi editor use :wq!).

Step 2. Installing required packages
this is probably the easiest step of the whole write up and you have probably already done it before looking up this post...
Navigate to System -> Package Manager -> Available Packages

Now look for Squid, SquidGuard and Lightsquid (if you want a log phraser)There is a small bug whit squid and SquidGuard installation that I have seen a few times and that is that you need to install the packages in a certain order for them to work properly

1. Squid
2. Lightsquid
3. SquidGuard



When installation is done you are done whit step 2.


Step 3. Configuration of Squid
Now we are going to setup the Squid service to handle the all the HTTP and HTTPS traffic for our clients, but before we can start the configurations Squid have an little bug where it will not save any of your settings before the Local Cache values are set so navigate to Services -> Squid Proxy Server -> Local Cache then set whatever options you like or scroll down to the bottom and hit save.

When the site refreshes from saving the Local Cache settings navigate to Services -> Squid Proxy Server -> General Settings
I have the following option set:

Squid General Settings
Enable Squid Proxy: Yes
Keep Settings/Data: Yes
Proxy Interface(s): LAN & Loopback
Proxy Port: 3128 (you can change this to a custom one if you like)
Allow Users on Interface: Yes



Transparent Proxy Settings
NO I do not use this leave option empty



SSL Man In the Middle Filtering
HTTPS/SSL Interception: Yes
SSL/MITM Mode: SPLICE ALL <- THIS IS AN IMPORTANT SETTING, IF SETT WRONG IT WILL NOT WORK.
SSL Proxy Port: 3129 (you can change this to a custom one if you like)
SSL Proxy Compatibility Mode: Modern
DHParams Key Size: 2048 Default
CA: SquidCA <- This is the rootCA you created in Step 1.
SSL Certificate Daemon Children: 5 Default
Remote Cert Checks: Do not verify remote certificates
Certificate Adapt: Sets the "Not Before" (setvalidbefore)



Logging Settings
Enable Access Logging: Yes
Log Store Directory: /var/squid/logs
Rotate Logs: 62 - keeps 2 months of logs in case of access reviews or issues, large SSD recommended
Log Pages Denied by SquidGuard: Yes
Save and apply



Step 4. Configuration of SquidGuard Proxy filter
This is where you define your ACL's and Blacklist, I do not use any pre-defined blacklist in this guide as I believe you get better control when you set it up manually from scratch, I am going to use Facebook and YouTube as primary targets to block as these are the most requested sites to be blocked by my clients, but this will work for any sites running on HTTP and HTTPS.

To start the configuration navigate to Services -> SquidGuard Proxy filter -> General Settings

General Options
Enable: Yes

LDAP Options
NO I do not use this leave option empty

Logging options
Enable GUI log: Yes
Enable log: Yes
Enable log rotation: Yes

Miscellaneous
Clean Advertising: Yes

Blacklist options
NO I do not use this leave option empty
Save and apply.





Now that SquidGuard is configured and running we need to setup some instructions for it to follow in terms of what to allow and what to block, this is called Target Categories or Target ACL's you can configure this by navigating to Services -> SquidGuard Proxy filter -> Target Categories

There is a bug in SquidGuard that it will not initiate the blacklist blocking before it has a dummy ACL defined under Target Categories so we need to create 3 ACL's for this to work properly



1. Dummy
2. myBlockList
3. myAllowList

Dummy ACL
Name: Dummy
Description: Dummy ACL
Save





myBlockList
Name: myBlockList
Order: ---
Domain List: facbook.com fb.com youtube.com
URL List: facbook.com/ fb.com/ youtube.com/
Regular Expression: BLANK
Redirect mode: int error page
Redirect: these sites have been blocked by your ADMIN, if you have business reason to visit this page contact your supervisor.
Description: Blocked internet sites
Save





myAllowlist
Name: myAllowlist
Order: ---
Domain List: it-monkey.net company.local
URL List: it-monkey.net/ company.local/
Regular Expression: BLANK
Redirect mode: None
Redirect: BLANK
Description: All allowed sites to bypass Proxy filter
save





You should now have all the needed Target ACL's configured to block Facebook and YouTube, but you may wonder why you put the information in both the Domain List and URL List option of the ACL's and there is a reason for that. If a site uses HTTPS it will read from the Domain List option and if it uses HTTP then it uses URL List option and it the sites uses both then you need to have it in both places to fully block the site

HTTP = URL List
HTTPS = Domain List


Now we need to assign an action for what SquidGuard should do whit the different "Categories" this is where you specify if the list you created is a Blacklist and should be blocked or if it’s a whitelist and should bypass all the filters and always allow traffic for it, to do this navigate to
Services -> SquidGuard Proxy filter -> Common ACL

General Options
Target Rules List  + \ -
[Dummy]  Access: ---
[myBlockList]  Access: Block
[myAllowlist]  Access: Whitelist
Default access [all] Access: Allow



Do not allow IP-Addresses in URL: Yes
Proxy Denied Error: Default
Redirect mode: int error message
Use SafeSearch engine: Yes
Rewrite: None
Save and apply



Now that the entire configuration in Squid and SquidGuard is done you need to apply it to the current running configuration by pressing the large green Apply button has to be pressed found on
Services -> SquidGuard Proxy filter -> General Settings

" Important: Please set up at least one category on the 'Target Categories' tab before enabling. The Save button at the bottom of this page must be clicked to save configuration changes. To activate SquidGuard configuration changes, the Apply button must be clicked. "

Now we have to setup WPAD and Firewall rules for pfSense to automatically push the Proxy configuration to its client and also who has access to connect and use the internet on your network, I will setup WPAD first since it will only work when pfSense GUI runs on HTTP and not the default HTTPS so it will affect how you configure your firewall rules.

Step 5. Configuring pfSense to act as WPAD for Squid
For security purpose I am separating the WebGUI and the WPAD servers by using a custom port HTTP TCP port for the WebGUI, I will be running WebGUI on 8080 and WPAD on 80
(8080 is a known admin\ gui port and is only used as example; you should set a custom port for your network)

Stat by creating a allow rule so you don’t lock yourself out of the firewall by going to Firewall -> Rules -> LAN and create an allow rule for port 8080

Action: Pass
Disabled: NO
Interface: LAN
Address Family: IPv4
Protocol: TCP
Source: Single host or alias: IT_Department
Destination: This Firewall (Self)
Destination Port Range: From: other Custom  8080 To: other Custom  8080 
Description: Allow IT-Admins access to WebGUI



I use Aliases for everything, but if you do not have an IT-Department Alias defined, then just set the source to the IP-Address of your main computer.
Save and apply this rule



Now we are ready to change the WebGUI to a custom HTTP port without locking yourself out, to do this navigate to System -> Advanced -> Admin Access

webConfigurator
Protocol: HTTP
TCP port: 8080
Save and apply, then wait for it to automatically redirect your session to the new port this takes about 30sec or so.



Next thing you need to do is set the DNS Record for WPAD to resolve to the webserver that will be hosting your Proxy settings file, since I will run all of this out of pfSense I use the DNS Resolver service for this, If you want you could use whatever DNS and Web server you want as long as you set it to resolve the correct URL to its correct IP.
Navigate to Services -> DNS Resolver add a new Host Override

Host Override Options
Host: wpad
Domain: it-monkey.local  (Set your own internal domain here, if you do not have one you can set one in the System -> General Settings)
IP Address: 192.168.1.1  (LAN IP of pfSense, this has to be set to the local IP of the interface you want to run the Proxy on)
Description: WPAD Autoconfigure Host
Save and Apply



General DNS Settings



Internal Domain Settings



Now we are ready to create the actual client configuration file that will be pushed by this setup, some programing is involved so open your Notepad++ and get ready to create the following 3 files:

wpad.dat
wpad.da
Proxy.pac

All of these files require the same code so it is just a copy\paste or save-as exercise, the code needed is

function FindProxyForURL(url,host)
 {
 return "PROXY IP-of-pfSense-LAN:PORT-of-Squid";
 }

In my setup that would be

function FindProxyForURL(url,host)
 {
 return "PROXY 192.168.1.1:3128";
 }



Now you would need to upload these files to pfSense, I recommend using WinSCP or similar for this the path you want to store these files in are:
/usr/local/www

When all files are uploaded you should see them in the directory like

/usr/local/www/wpad.dat
/usr/local/www/wpad.da
/usr/local/www/Proxy.pac

Now that all this is done we are done whit the WPAD configurations and only need to adjust our firewall rules and lockdown unrestricted access of our network.

Step 6. Logging whit Lightsquid
Lightsquid is currently the only supported and maintained log phraser that you can use whit Squid and SquidGuard on pfSense, this is a small applicatuon that takes all the logs from your proxy server and sort them by your prefrense and the present them to you in a easily read format through its own webinterface. To configure this serverce navigate to Status -> Squid Proxy Reports

Web Service Settings
Lightsquid Web Port: 7445 (Default port, you can use custom port)
Lightsquid Web SSL: Yes
Lightsquid Web User: Admin
Lightsquid Web Password: MysuperSecretandSEXYkeYg3n3ratedbyN0tApA$$w0rdgenetAT0r

Report Template Settings
Language: English
Report Template: Base
Bar Color: Orange

Reporting Settings and Scheduler
IP Resolve Method: DNS
Skip URL(s): None left blank
Refresh Scheduler: None




Step 7. Configuring the Firewall Rules
Start by navigating back to Firewall -> Rules -> LAN
Depending on your preferences you should only need to have about 5 - 8 rules in this list, I currently only have 6 rules defined and in use, those are the following in order

Allow IT Department management access to pfSense
Allow ICMP form LAN Clients
Allow DNS from LAN Clients
Allow WPAD from LAN Clients
Allow Proxy from LAN Clients
Block Everything else from anywhere



Whit this rule set only Ping and DNS traffic is allowed to be used outside of the Proxy filter, this is a good thing for diagnostics purpose in case something breaks for your clients , you can use simple tests for connectivity using Ping and DNS without touching the Proxy or Firewall settings

The rules are defined whit the following details

Allow IT Department management access of pfSense
Action: Pass
Disabled: NO
Interface: LAN
Address Family: IPv4
Protocol: TCP
Source: Single host or alias: IT_Department (alias IT_Department contains IP of 192.168.1.2 which is the main desktop used by IT)
Destination: This Firewall (Self)
Destination Port Range: From: other Custom  PF_MGMT To: other Custom  PF_MGMT  (port alias contains port 22, 7445 and 8080 and allow you to define it in a single rule)
Description: Allow IT Department management access of pfSense



Allow ICMP form LAN Clients
Action: Pass
Disabled: NO
Interface: LAN
Address Family: IPv4
Protocol: ICMP
ICMP Subtypes: Any
Source: LAN Net
Destination: Any 
Description: Allow ICMP



Allow DNS from LAN Clients
Action: Pass
Disabled: NO
Interface: LAN
Address Family: IPv4
Protocol: TCP \ UDP
Source: LAN Net
Destination: ANY
Destination Port Range: From: DNS To: DNS
Description: Allow DNS



Allow WPAD from LAN Clients
Action: Pass
Disabled: NO
Interface: LAN
Address Family: IPv4
Protocol: TCP
Source: LAN Net
Destination: This Firewall (Self)
Destination Port Range: From: other Custom  WPAD To: other Custom WPAD  (port alias WPAD contains port 80 \ HTTP)
Description: Allow WPAD



Allow Proxy from LAN Clients
Action: Pass
Disabled: NO
Interface: LAN
Address Family: IPv4
Protocol: TCP \ UDP
Source: LAN Net
Destination: This Firewall (Self)
Destination Port Range: From: other Custom  3128 To: other Custom  3129
Description: Allow Proxy



Block Everything else from anywhere
Action: BLOCK
Disabled: NO
Interface: LAN
Address Family: IPv4
Protocol: ANY
Source: ANY
Destination: ANY
Destination Port Range: ANY
Log: Yes
Description: Block Everything



Firewall Aliases

PF_MGMT Ports



WPAD



IT_Admin



All Aliases



When you have all these rules created and applied then all the pieces are in place and you are ready to test these settings on your client computer, However since you have done a lot of configurations as changes to the pfSense systems and rules I suggest that you reboot it so you clear out any conflicting rule or state stuck in the system memory, that way you would start testing on a freshly booted system and rule set.

If you follow this guide and it is not working for you and it broke your system, I am not responsible or liability for that as you should not take anything you read on the internet at face value and you should test settings like this in a lab environment and not on your production servers.

10
Edd Noman's Guide to pfSense 04 – How-To Block Ad’s and Websites using pfBlockerNG

In this guide I will be covering how to use the DNSBL feature of pfBlockerNG package to block users from accessing unwanted websites like porn, Facebook or YouTube and also keep your users safe from known infected website so that the risk of getting infected whit viruses or malware is reduces, this will also clean up Ad’s seen on websites also so you get a better browsing experience.

How pfBlockerNG and DNSBL achieve to do all this even if the sites uses HTTPS and SSL encryptions is by using DNS bases aliases that has both the Domain and IP to generate the firewall rules. These aliases are generated upon predefined txt files that contain the IP and Domain information that are updated by known security professionals and provider once a bad IP or Domain is identified.

The only issue whit pfBlockerNG and DNSBL is that it can use a lot of resources both RAM and CPU the more lists you assign it the more RAM and CPU it would need to process all of them, the lowest set of hardware I would recommend using for this is 2Gb RAM and 4Core 1.5Ghz processor.

pfBlockerNG uses the DNS Resolver service of pfSense to handle DNS resolutions so before we start the installation make sure your DNS Resolver is running whit the Forwarding mode enabled, this is found under Services -> DNS Resolver -> General Option



Now you would install pfBlockerNG as you would any other package by navigating to: System -> Packet Manager -> Available Packets and then search for “pfBlocker” and then click install



Confirm that you want to install the package and all its dependency



Now the installer and progress bar will go over your screen, give it a few moments to complete as it is a fairly large package to download and install and depending on the system you using it can take a few moments to get it done



When installation is done Navigate to Firewall -> pfBlockerNG to start the configuration

General Settings
Enable pfBlockerNG: Yes (checked)
Keep Settings: Yes (checked)
CRON Settings: Every hour | :15 | 0 | 0 (this will sync the list every 15min past a full hour ie, 01:15 then 02:15)
De-Duplication: Yes (checked)
CIDR Aggregation: Yes (checked)
Suppression: Yes (checked)
MaxMind Localized Language: English
Download Failure Threshold: 4
Logfile Size: 20000



Interface/Rules Configuration
Inbound Firewall Rules: WAN | Block
Outbound Firewall Rules: LAN | Reject
Rule Order: pfB_Pass/Match | pfB_Block/Reject | pfSense Pass/Match | pfSense Block/Reject
Auto Rule Suffix: Auto Rule
Kill States: Yes (checked)
Save and Apply



Now we want to move on to DNSBL settings found on Firewall -> pfBlockerNG -> DNSBL

DNSBL Configuration
Enable DNSBL: Yes (checked)
Enable TLD: Yes (checked)
DNSBL Virtual IP: 10.10.10.1
DNSBL Listening Port: 8081
DNSBL SSL Listening Port: 8443
DNSBL Listening Interface: LAN
DNSBL Firewall Rule: Yes | LAN



DNSBL IP Firewall Rule Settings
List Action: Deny Outbound
Enable Logging: Enable
Save and Apply



Next we configure the DNSBL EasyList from Firewall -> pfBlockerNG -> DNSBL -> DNSBL EasyList

DNSBL – EasyList
DNS GROUP Name: EasyList
Description: EasyList

EasyList Feeds
State: ON | EasyList Feeds: EasyList W\O Elements | Header: EasyList
State: ON | EasyList Feeds: EasyPrivacy | Header: EasyPrivacy



DNSBL - EasyList Settings
Categories: All selected*
List Action: Unbound
Update Frequency: Once a day
Weekly (Day of Week): Monday
Save and Apply



Now we get to the heart of the configurations as we need to define our DNSBL Feeds and from where pfBlockerNG should pull its information from, a good source for this is on the pfSense forum from the developer himself on this link: https://forum.pfsense.org/index.php?topic=102470.msg573167#msg573167

Navigate to Firewall -> pfBlockerNG -> DNSBL -> DNSBL Feeds to add your feeds lists
I am currently running 4 different feeds and those are: Adverts, Malicious, DGA Crypto and hpHost and I have them configured whit the following information

Adverts
DNS GROUP Name: Ads
Description: DNSBL Adverts
DNSBL:
Format: Auto|State: ON | Source: http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext | Header: yoyo
Format: Auto|State: ON | Source: http://hosts-file.net/ad_servers.txt | Header: hpHosts_ads
Format: Auto|State: ON | Source: https://adaway.org/hosts.txt | Header: Adaway
Format: Auto|State: ON | Source: http://sysctl.org/cameleon/hosts | Header: Cameleon
List Action: Unbound
Update Frequency: Every 8hours
Weekly (Day of Week): Monday
Save and Apply



Malicious
DNS GROUP Name: Malicious
Description: DNSBL Malicious
DNSBL:
Format: Auto|State: ON | Source: http://hosts-file.net/download/hosts.zip| Header: hpHosts
Format: Auto|State: ON | Source: http://someonewhocares.org/hosts/hosts| Header: SWC
Format: Auto|State: ON | Source: https://raw.githubusercontent.com/Dawsey21/Lists/master/main-blacklist.txt| Header: spam404
Format: Auto|State: ON | Source: https://malc0de.com/bl/BOOT| Header: malc0de
Format: Auto|State: FLEX | Source: https://mirror1.malwaredomains.com/files/justdomains | Header: MDS
Format: Auto|State: ON | Source: http://winhelp2002.mvps.org/hosts.txt| Header: MVPS
Format: Auto|State: ON | Source: http://www.malwaredomainlist.com/hostslist/hosts.txt| Header: MDL
List Action: Unbound
Update Frequency: Once a day
Weekly (Day of Week): Monday
Save and Apply



DGA
: DGA
Description: DNSBL DGA for Cryptolocker
DNSBL:
Format: Auto|State: ON | Source: http://osint.bambenekconsulting.com/feeds/dga-feed.gz| Header: BBC_DGA
Format: Auto|State: ON | Source: http://osint.bambenekconsulting.com/feeds/c2-dommasterlist.txt| Header: BBC_C2
List Action: Unbound
Update Frequency: Every 8hours
Weekly (Day of Week): Monday
Save and Apply



hpHost
DNS GROUP Name: hpHosts_partial
Description: DNSBL hpHosts_partial
DNSBL:
Format: Auto|State: ON | Source: http://hosts-file.net/hphosts-partial.asp| Header: hpHosts_partial
List Action: Unbound
Update Frequency: Every 6hours
Weekly (Day of Week): Monday
Save and Apply



When done you should be left whit the same 4 categories for blocking Ad’s and some of the malware and crypto locker, you can add more list and sources to this configuration but you need to do your own research on that. The entire list I used is free and open source maintained but there is also paid alternatives for feeds.



At this point we have only configured pfBlockerNG to use DNSBL and react on domain names, but we would also want to block the known bad IPs out on the internet to do this navigate to: Firewall -> pfBlockerNG -> IPv4

I will only be covering IPv4 in this guide, but if you use IPv6 the same method is used to add the rules needed for that, I just do not have the source list to provide you. My IPv4 alias list is called Badguys and are configured as following

Badguys
Alias Name: Badguys
List Description: IPv4 Badguys
IPv4 Lists:
Format: Auto|State: ON | Source: https://gist.githubusercontent.com/BBcan177/d7105c242f17f4498f81/raw/90eb2ac8bdc01af3008d728b7c0f10dc7b2506b4/MS-3| Header: BBcan177_Domains_IPv4
Format: Auto|State: ON | Source: https://rules.emergingthreats.net/blockrules/compromised-ips.txt| Header: ETCompromised
Format: Auto|State: ON | Source: https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt| Header: ETBlocked
Format: Auto|State: ON | Source: https://gist.githubusercontent.com/BBcan177/bf29d47ea04391cb3eb0/raw/b344ebc9475acdea1fae38a12c4ea9332838a184/MS-1| Header: BBcan177Threats
Format: Auto|State: Auto | Source: http://www.malwaredomainlist.com/hostslist/ip.txt| Header: Malwaredomainlist
Format: Auto|State: ON | Source: https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt| Header: Ransomware
List Action: Deny Both
Update Frequency: Once a day
Weekly (Day of Week): Monday
Enable Logging: Enable
States Removal: Enable
Save and Apply





And this is all that is needed for a working setup of pfBlockerNG to act on known bad sites ether whit domain or IP, it will also block AD’s on webpages you visits, and the last thing for us to do is creating a custom block list of website we do not want our user to visit even though they are marked as clean you may have other reasons for why they need to be blocked. This is handled by the TLD feature of DNSBL so to do this we need to navigate to: Firewall -> pfBlockerNG -> DNSBL

At the bottom of this page you have several advanced options the one we are after is TLD Exclusion List, TLD Blacklist and TLD Whitelist. Yes the names are as intuitive as to what functions they have

TLD Exclusion List will be used for domains you do not want to be included in any pfBlockerNG rules or aliases, no action is performed on these domains, other rules in pfSense may apply
TLD Blacklist will be used for domains you want to specifically block access to
TLD Whitelist will be used for any domains you want to always be accessible, these sites will not be blocked by the firewall.

Open up TLD Blacklist and enter the following domains:

fb.com
facebook.com
youtube.com
cnn.com

Save



After you have made any changes to DNSBL or the TLD lists you need to force an update of the new rule set before the changes take effect on your network, to do a update navigate to Firewall -> pfBlockerNG ->Update
When doing a manual update or forced update due to changes in the config you do not want to run it close to when the cron task is running there is a clock on the update page that will tell you when the next update will run

Select 'Force' option: Reload
Select 'Reload' option: All



When the update is finished you can try to access one of the domains you blocked in TLD Blacklist and you should only get a black screen instead of the actual site loading





Now this should be all there is to blocking websites and ad’s from pfBlockerNG

If you follow this guide and it is not working for you and it broke your system, I am not responsible or liability for that as you should not take anything you read on the internet at face value and you should test settings like this in a lab environment and not on your production servers

11
Edd Noman's Guide to pfSense 03 - How-To Monitor Bandwidth Usage whit NtopNG

In this guide I will only focus on bandwidth monitoring whit use of the NtopNG package in pfSense, bandwidth monitoring is a complex topic on its own and I will try to provide the basic you need to understand and get started whit this task and how this can improve your network situation. pfSense have several option for monitoring bandwidth and you can read about them here: https://doc.pfsense.org/index.php/How_can_I_monitor_bandwidth_usage

A word on bandwidth monitoring:
While it used to be that monitoring your bandwidth meant solely focusing on internet traffic, bandwidth usage monitoring now encompasses a broader range of components. For example, you can monitor bandwidth speed or capacity, you can observe network traffic between devices or general web application traffic. Regardless of what traffic you are monitoring, though, its important to understand the bandwidth that is being utilized so you can ensure users are getting the best possible performance out of your network.

I would go as far as saying that monitoring your network bandwidth usage is the most critical function for any network administrator.

What does bandwidth really mean?
To sum it up in one word: data. Bandwidth is quantified as the amount of data transferred in time, typically measured in bits per second. Thirty years ago, data was sent through physical mediums like the postal service; now there are myriad ways to transmit and receive massive amounts of data with the push of a button.

Since most organizations rely on the internet to conduct business-critical operations, internet speed can make all the difference in their success. What people dont know, however, is that there are actually two different types of bandwidth speed: upload and download. Upload speed is the speed at which data is sent to its destination, while download speed refers to the rate at which data is received. It used to be normal for businesses to use low bandwidth services like 56.k modems to transmit information. Now, those who have the funds can install Gigabit speed to their infrastructure to support the growth in data consumption. Firms are still using DSL and cable connections to run their business, but service providers have been able to allocate more resources to support these lower-tier Telco options.
 
Bandwidth capacity is also an important consideration. Bandwidth capacity means the maximum data rate a link can transfer. Bandwidth capacity is an important factor to consider, because when you configure your infrastructure you need to make sure you can support the bandwidth that you require. For example, the service model of your cell phone plan is based on how much data you consume on the vendors network. When you're able to monitor your bandwidth usage, you're better able to determine what plan is right for your environment. As another example, consider an environment with hundreds of users. How do you determine what bandwidth to implement? Its important to have proper metrics and tools that can show you how much bandwidth you'll require to run your day-to-day operations. For business customers, it can become very difficult to forecast current and future consumption of bandwidth without network insight.

High-Speed Web-based Traffic Analysis and Flow Collection whit NtopNG
Ntopng is the next generation version of the original ntop, a network traffic probe that shows the network usage, similar to what the popular top Unix command does. ntopng is based on libpcap and it has been written in a portable way in order to virtually run on every Unix platform, MacOSX and on Windows as well.

Ntopng users can use a a web browser to navigate through ntop (that acts as a web server) traffic information and get a dump of the network status. In the latter case, ntopng can be seen as a simple RMON-like agent with an embedded web interface. The use of:
a web interface.
Limited configuration and administration via the web interface.
Reduced CPU and memory usage (they vary according to network size and traffic).

Features of Ntopng:
- Sort network traffic according to many criteria including IP address, port, L7 protocol, throughput, AS.
- Show network traffic and IPv4/v6 active hosts.
- Produce long-term reports about various network metrics such as throughput, application protocols
- Top X talkers/listeners, top ASs, top L7 applications.
- For each communication flow report network/application latency/RTT, TCP stats (retransmissions, packets OOO, packet lost), bytes/packets
- Store on disk persistent traffic statistics in RRD format.
- Geolocate hosts and display reports according to host location.
- Discover application protocols by leveraging on nDPI, ntop's DPI framework.
- Characterize HTTP traffic by leveraging on characterization services provided by Google and HTTP Blacklist.
- Show IP traffic distribution among the various protocols.
- Analyze IP traffic and sort it according to the source/destination.
- Display IP Traffic Subnet matrix (who's talking to who?)
- Report IP protocol usage sorted by protocol type.
- Produce HTML5/AJAX network traffic statistics.
http://www.ntop.org/products/traffic-analysis/ntop/

Whit all that said, we can start by installing the ntopng package by navigating to System -> Package Manager -> Available Packages
Here you search for "ntop" and then click the install button



Now you would need to confirm that you would like to install selected package and its dependency



Now you will see the install process starting whit the progress bar and text output will run for some minute as the installer is about 250mb that would need to be downloaded and then installed so give it some time



When the package is done installing you need to navigate to Diagnostics -> ntopng Settings
The first thing you would want to do here is to set the password to use when login to the ntopng WebGUI

ntopng Admin Password: MysuperSecretandSEXYkeY
Confirm ntopng Admin Password: MysuperSecretandSEXYkeY
Save and Apply



Now after the access password is set we can configure the default settings for ntopng to use

General Options
Enable ntopng: Yes
Keep Data/Settings: Yes (this will keep all settings in case of updates and reinstallations)
Interface: LAN and WAN
DNS Mode: Decode DNS responses and resolve all numeric IPs
Local Networks: Consider all RFC1918 networks as local traffic (this depends on your layout of pfSense is behind NAT you want to use LAN Interface as local)
Disable Alerts: Yes
Save and Apply



When ntopng service is configured and have start running after its done loading when pressing the save button you want to update the GeoIP Data information so click on the green button for updating the information
At this point the configuration in pfSense is done, and you now need to connect to the ntopng WebGUI to finish its configuration so navigate to https://192.168.1.1:3000/ or IP-OF-pfSense:3000 or if you have followed my previous install guides https://10.99.99.1:3000



Once logged in, the first place to go is Settings and set your recording limits, this is done whit the gear icon on the top right corner



Out of the box, it will record RAW packets for 1 day in your File System, the Rolled up reports in MySQL for 30 days, and Total's for 1 year. You can adjust here to work with the available disk space and RAM you have for pfSense.







The Reports:
Everyone will be different and have their own needs for reporting, but I wanted to screenshot some of the cool reports you can generate and view in ntopng to share with you all.
You can also customize and work with anything which is captured going across the LAN.
You can view total traffic on your local network and sort by usage:



You can view Active Data Flows / Destination / Type in real time with ease on the Active Flows Report



You can even view specifics on a specific host on your Network like so, even with total usage, Activity Maps and more



Even break it down by Protocols



For those of you who are more interested in spying, yes you can see the top HTTP traffic destinations....



Now that is all there is to configure ntopng and where to find the different reports, and yes there is a large benefit to running this package even in your home environment as this will give you graphs and statistics of where all your data and bandwidth went and what device used it, and if you did not get the amount you charged for you have actual proof of your usage when you call your ISP to complain about their service and invoice.

If you follow this guide and it is not working for you and it broke your system, I am not responsible or liability for that as you should not take anything you read on the internet at face value and you should test settings like this in a lab environment and not on your production servers

12
Thank you,

Yes the mailreport package can only send the logs that are local on pfSense, but you can use external syslog server whit pfSense and then have the syslog server email you whit the same report, this would be the preferd way of doing things like that but that is out of scope for this guide as it is for improved network functionality whit basic features of pfSense and not how-to configure a syslog server and export pfSense logs to it

13
There is a lot more sources to renewable power than solar, but since Nevada and Las Vegas is a desert it make sense to use solar technology rather than wind or water, here in Norway 90% of our electrical power is from water from our hydroelectric power plants if you have the opportunity then I suggest there is always best to could provide for your self so setting up your own little power plant would be a good idea

14
Edd Noman's Guide to pfSense 02 - How-To Improve Network Functionality whit Basic Features of pfSense 2.3

This is a follow up guide to my prevourius guide: How-To Install and Configure pfSense 2.3 inside a VM using VMware workstation I will suggest you give that a look before you continue on this as that will give you a common reference on the network layout used in this guide, since this guide starts whit a fresh install of pfSense anyone can follow along but I will use the layout referenced in my last guide for consistency purpose.
In this guide I will show you some of the basic features or services of pfSense 2.3 and how-to take advantage of them to improve your overall experience and ease of use for the LAN network.

The features and services I will be covering in this guide will be the following:
- Remote Access whit use of Firewall Aliases and Firewall Rules
- NTP Services for clock synchronizations of your LAN devices
- UPnP Service for those who use Game consoles and are stuck whit Strict-NAT error
- DNS Resolver as local DNS Server
- Dynamic DNS whit DYN or NO-IP services
- DHCP Server and Static IP assignments
- Email reports and alerts

This is again only the basic settings and configurations of said features and services, as this guide series is not a comprehensive guide on how networking or pfSense functions, do not expect a full explanations of each feature settings, I will however try to cover the most important settings for you configure for the services to function properly.

1. Remote Access and Management of pfSense
The reason you may want to configure Remote Access to your firewall is that you might not always be at the location whit physical access to your firewall, and something has happened that you need to address right away, then Remote Access is what allows you to connect from anywhere in the world from any device that has a web-browser and internet connectivity.

There are several ways of doing this but the one I am going to cover here is called Restricted Firewall Access, which means that we only open up access from known IPs and Subnets and uses a custom ports for the WebGUI and SSH Consoles of the firewall, if you are interested in other way of doing Remote Access the pfSense doc covers it here: https://doc.pfsense.org/index.php/Remote_firewall_Administration
Lets start whit changing the default ports of the WebGUI and SSH by navigate to: System -> Advanced -> Admin Access

Here you set the following options
Protocol: HTTPS
TCP Port: 8080
Anti-Lockout rule: Yes (Disabled - checked)
Secure Shell Server: Yes (Enabled - checked)
SSH Port: 2222
Save and Apply

Warning: The ports 8080 and 2222 for WebGUI and SSH is well-known managements ports by administrators and hackers and I only use them for guide purposes and you should select ports that matches your network preferences.



Now you would need to wait about 30sec for it to apply the new settings and redirect and refresh your web-browser to the new port,  your new method of accessing pfSense would be by using the following web address: https://10.99.99.1:8080 from the LAN interface.

Now that all the custom ports are set we need to create some aliases so that we can define it in our firewall rules, aliases are just like what it sounds like it is a collection of IP, Ports or URL that is given a distinctive name so that it helps us to create more human readable rules, you can get more information about Aliases in the pfSense documentation here: https://doc.pfsense.org/index.php/Aliases
To create aliases we navigate to: Firewall -> Aliases

Here we want you need to create 2 different aliases, 1 Port alias and 1 IP alias to be able to create our Remote Access rules
Port Aliases is created from:Firewall -> Aliases -> Ports

Name: MGMT_Ports
Description: pfSense Management ports
Type: Port(s)
Port: 2222, 8080
Save



IP Aliases is created from: Firewall -> Aliases -> IP
When creating IP based aliases you can choose between 2 types of aliases Host(s) based or Network(s) based alias, The main difference between them is:

Host(s) based:  You need to specify each IP Address you want in a separate entry, this is good to use for a small internal network alias where you want the rule only to apply to a small number of IPs
Network(s) based: You can specify whole network segments and subnets as a single entry, but you can also specify a single IP Address. 

Name: RemoteAdmin
Description: Remote Administrators
Type: Network(s)
Network or FQDN: 192.168.1.199/32 or 192.168.1.0/24

If I only want access from my computer whit IP 192.168.1.199 I need to set that as a /32 network and all single devices need to be specified by IP-address+/32 in the alias, this should only be used if you have a static IP address on the computer as if the IP change after a reboot or something you will no longer have access to your firewall.

If I would like to give every one that connects to the D-Link Home-Router access to connect to pfSense I need to specify the Home-Router LAN network of 192.168.1.0/24 in this alias, whit this set anyone whit the IP-address from 192.168.1.1 to 192.168.1.254 will be able to connect to your firewall and manage it.
As you can see in the picture below I have added both 192.168.1.199/32 and 192.168.1.0/24 and only 1 of them is really needed as 192.168.1.199 is part of the 192.168.1.0/24 network I only added it for comparison reason.



Since I trust my own Home Router network the alias would be

Name: RemoteAdmin
Description: Remote Administrators
Type: Network(s)
Network or FQDN: 192.168.1.0/24
Save and Apply



Now that our ports and aliases are set and created we are ready to create the actually rules to allow the traffic to pass and connect to the firewall, to do this navigate to: Firewall -> Rules -> WAN

Action: Pass
Disabled: NO
Interface: WAN
Address Family: IPv4
Protocol: TCP
Source: Single host or alias:  RemoteAdmin
Destination: This Firewall (Self)
Destination Port Range: From: other Custom  MGMT_Ports To: other Custom  MGMT_Ports
Log: Yes
Description: Allow RemoteAdmin access to pfSense
Save and Apply





Now when you have saved and applied the rules you should be able to access your firewall from the Home-Router LAN network and you should see the following Firewall Rule list for WAN interface



This is all there is to setup Remote Access and Management of your firewall, Just a word of warning if you intend to do this when directly connected to internet and not behind your ISP or Home-Router you need to be very specific of what IP addresses and networks you allow to connect and you should use a custom username and strong password.

2. NTP Services for Clock Synchronizations of your Devices
The NTP Service (ntpd), is configured at Services -> NTP, allows pfSense to act as a Network Time Protocol server for a network, and also keeps the clock in sync against remote NTP servers as an NTP client itself.
Before enabling this service, ensure that the router's clock keeps fairly accurate time. By default the NTP server will bind to and act as an NTP server on all available IP addresses. This may be restricted using the Interface(s) selection on Services -> NTP.

This service should not be exposed publicly. Ensure inbound rules on WANs do not allow connections from the Internet to reach the NTP server on the firewall.
https://doc.pfsense.org/index.php/NTP_Server

When you are to configure this you want the NTP servers you are using to be as close and local to you as possible for you to have the most accurate overall time, to find the best NTP Servers I use the NTP POOL Project website https://pool.ntp.org

To keep this guide as generic  as possible I am going to use the following servers:
0.pool.ntp.org
1.pool.ntp.org
2.pool.ntp.org
3.pool.ntp.org

But as stated you should find the ones that are closest to where you actually are, whit that said lets start configuring it by navigating to: Service -> NTP

NTP Server Configuration:
Interface: LAN
Time Servers:  0.pfsense.pool.ntp.org, 0.pool.ntp.org, 1.pool.ntp.org, 2.pool.ntp.org, 3.pool.ntp.org
Orphan Mode: 12
NTP Graphs: Yes (Enabled checked)
Save and Apply





Now it will use some time to synchronize the clock of pfSense and setup the service to serve your network the correct time, You can check the NTP Server status by navigating to: Status -> NTP



And that is all it is to setting up NTP Service of pfSense.

3. UPnP and NAT-PMP Service for Game consoles
What are UPnP and NAT-PMP?
UPnP is short for Universal Plug and Play and is commonly found on Windows, BSD, Linux systems and common home routers.
NAT-PMP is short for NAT Port Mapping Protocol and is similar to UPnP but found more commonly on Apple devices and programs. A growing number of programs support both methods. pfSense supports both, and the service may be configured at Services -> UPnP & NAT-PMP.

UPnP and NAT-PMP both allow devices and programs that support them to automatically add dynamic port forwards and firewall entries. The most common uses are in gaming systems (XBox, PlayStation, etc) and Bit Torrent programs like µTorrent, which both rely on allowing inbound connections to a local service.

There are some potential security risks using this feature like:
When UPnP or NAT-PMP are enabled, use only devices and programs which are trusted. These mechanisms will allow these entities to bypass the firewall to allow incoming connections with no additional control or authorization. Do not be surprised when this happens.
Access permissions for the service may be crafted in the options on pfSense. The format of these is shown in the GUI at Services > UPnP & NAT-PMP in the User specified permissions boxes. Using these, access could be restricted to a specific workstation or device.
https://doc.pfsense.org/index.php/What_are_UPnP_and_NAT-PMP

To configure UPnp & NAT-PMP navigate to:  Services -> UPnP & NAT-PMP

UPnP & NAT-PMP Settings:
Enable:  Yes (checked)
UPnP Port Mapping:  Yes (checked)
NAT-PMP Port Mapping:  Yes (checked)
External Interface: WAN
Interfaces: LAN
Log packets: Yes (checked)
Default Deny: Yes (checked)



UPnP Access Control Lists:
ACL Entries: allow 1024-65535 10.99.99.150/32 1024-65535
Save and Apply



Whit these settings UPnP & NAT-PMP is enabled but only the LAN client whit IP address of 10.99.99.15 are allowed to use this feature to open and close networks ports as needed and should help against Strict-NAT or NAT Type 3 issues on game consoles like PlayStation and Xbox devices behind pfSense firewall

4. DNS Resolver and Dynamic DNS Configuration

DNS Resolver:
Unbound DNS Resolver is a validating, recursive and caching DNS resolver. It provides various modules so that DNSSEC (secure DNS) validation and stub-resolvers are possible. Unbound was integrated into the base system in pfSense 2.2. Unbound is also the default DNS Resolver.
https://doc.pfsense.org/index.php/Unbound_DNS_Resolver

We start by configuring DNS Resolver general options by navigating to: Services -> DNS Resolver

General DNS Resolver Options:
Enable
:  Yes (checked)
Listen Port: 53
Network Interfaces:  ALL
Outgoing Network Interfaces:  ALL
System Domain Local Zone Type: Transparent
DNSSEC:  Yes (checked)
DNS Query Forwarding:  Yes (checked)
DHCP Registration:  Yes (checked)
Static DHCP:  Yes (checked)
Save and Apply





Now whit the above settings you would have a working DNS Resolver setup, however one of the advantages and cool thing of having your own DNS Resolver service running in your network is that you can assign your own custom domain addresses for your devices, to illustrate this option I am going to convert the access address of pfSense to use the URL of router.vmlab.lan and firewall.vmlab.lan instead of having to remember 10.99.99.1 IP address for it, to do this we use the Host Override Option.

Host Override Options:
Host: router
Domain: vmlab.lan
IP Address: 10.99.99.1
Description: DNS Record for pfSense Server

Additional Names for this Host
Host: firewall
Domain: vmlab.lan
IP Address: 10.99.99.1
Description: DNS Record for pfSense Server
Save and Apply



Now you should be able to access your pfSense firewall using the following URL and web-addresses:
pfSense.vmlab.lan
router.vmlab.lan
firewall.vmlab.lan

You can test this form CMD or Terminal whit the commands
Windows: nslookup
MAC \ Linux: dig



Hopefully you now see a lot of advantages of using the DNS Resolver in your network.

Dynamic DNS Service:
Dynamic DNS (DynDNS), found under Services -> Dynamic DNS, will update an external provider with the current public IP address on the firewall. This keeps a constant DNS hostname, even if the IP address changes periodically. Whenever an interface changes in some way, DHCP lease renew, PPPoE logout/login, etc, the IP will be updated.

There are many free DynDNS services out there, and pfSense supports more than 15 different providers. In addition to the normal public services, pfSense also supports RFC 2136 DNS updates to DNS servers.
In currently supported versions of pfSense, the DynDNS client supports using multiple DynDNS and RFC 2136 clients. These can be used to update multiple services on the same interface, or multiple interfaces.

There are two tabs under Dynamic DNS, one for DynDNS providers, and one for RFC 2136 servers. Each tab has a list of currently configured clients, which reflects not only their configuration but also their status. Additional clients can be managed from these lists.
When editing a DynDNS client, first pick a DynDNS service provider, then choose Interface with the IP address to update. Enter a hostname, username, password, and description. Optionally, an MX record and wildcard support may be enabled depending on the provider.

When editing an RFC 2136 client, first pick the interface with the IP to update, enter a hostname, Time To Live (TTL) for the DNS record, Key name (which must match the setting on the server), Key type of Host, Zone, or User, an HMAC-MD5 key, the DNS server IP address, and a description. TCP transactions may optionally be used instead of UDP.
https://doc.pfsense.org/index.php/Dynamic_DNS

As for configuring this service it heavily depend on your your DDNS provider, however I have a demo account whit 2 of the most popular providers 1 is the free provider NO-IP(noip.com) and a paid alternative Dyn DNS(dyn.com) and most of these provider have their own instructions on how to set it up using different router clients, so when you have singned up for a Domain whit one of the provider and read through their configuration document you can navigate to Services -> Dynamic DNS

Dynamic DNS Client Example:
Disable: No
Service Type: Your provider, if it is not in the list select other
Interface to monitor: WAN
Hostname: Your full domain name whit the host record you created ie, firewall.mydomain.com
MX: this is only used in special cases where you have an email server behind pfSense
Wildcards: Depends on your provider
Verbose logging: Only used for troubleshooting
Username: Your username \ email that you use for login at your provider
Password: Your password or update key used at your provider
Confirm Password: Your password or update key used at your provider
Description: Your description of the config ie, Provider DNS Service

This is the most generic and non-provider specific example I can give anyone for this service, as this is just a update client and all the different providers have their own way of authenticate and verify your account. However I have specific instruction for NO-IP and DYN provider

NO-IP Dynamic DNS Client:
Disable: No
Service Type: NO-IP (free)
Interface to monitor: WAN
Hostname:  lablan.ddns.net
Username:  EddNoman (no you do not get my account details)
Password:  MysuperSecretandSEXYkeY  (no you do not get my account details)
Confirm Password:  MysuperSecretandSEXYkeY  (no you do not get my account details)
Description:  NO-IP DNS Service
Save & Force update



For NO-IP they use the email address (username) and password you created to sign up for their service, however for DYN DNS service you get assigned a Update API Key that you need for your password in the client



DYN Dynamic DNS Client:
Disable: No
Service Type: DynDNS (dynamic)
Interface to monitor: WAN
Hostname:  lablan2.qnett.net
Username:  EddNoman (no you do not get my account details)
Password:  MysuperSecretandSEXYkeY  (no you do not get my account details)
Confirm Password:  MysuperSecretandSEXYkeY  (no you do not get my account details)
Description:  DYN DNS Service
Save & Force update



As you can see from these pictures it is not hard to setup it is just a matter of filling in the correct provider information about your domain and account detail, also as you can see from the picture there is no issue whit having multiple providers and DNS entries pointing to the same router and firewall.



If you do not have any Dynamic DNS provider or account yet, I have a 15% off referral code for DYN DNS (dyn.com) that you can use, if you sign up for DYN DNS then I strongly recommend the Managed DNS Services as then you get your own fully fledged DNS Service whit all the features and benefits of your own public DNS Servers.
Referral Code:  RFE2X07RS8

5. DHCP Server and Static IP Assignments
The DHCP server in pfSense will hand out addresses to DHCP clients and automatically configure them for network access. By default, the DHCP server is enabled on the LAN interface.
The DHCP server page, found under Services -> DHCP Server, has a tab for each available interface. The DHCP daemon can only run on interfaces with a Static IP address, so if a tab for an interface is not present, check that it is enabled and set with a Static IP. The DHCP server cannot be active on any interface if the DHCP Relay service is in use.

DHCP Options
For each Interface, there are many options to choose from. At a minimum, the Enable box must be checked on the interface tab and an address range (starting and ending IP addresses) to use for DHCP clients must be defined. The other settings may be configured, but are optional. Each option is explained in more detail on the page and also in The pfSense Book.
See the DNS Forwarder article for information on the default DNS server behavior.
Some other options which may be set for clients include TFTP server, LDAP URI, and the ability to add in any custom DHCP option number and value.

Static IP Mappings
Static IP mappings can be added at the bottom of the DHCP server tab for a given interface.
To add a Static IP mapping, click "+", and then enter a MAC address, IP address, Hostname, and Description. These mappings can also be created from the DHCP Leases view. There are many other per-host options which can be sent as well. https://doc.pfsense.org/index.php/DHCP_Server

Lets define our network settings by configuring the DHCP Server, navigate to Services -> DHCP Server

General Options:
Enable: Yes (checked)
Subnet: 10.99.99.0
Subnet mask: 255.255.255.0
Available range: 10.99.99.1 - 10.99.99.254
Range: from 10.99.99.100 to 10.99.99.200



Servers
DNS server1: 10.99.99.1



Other Options
Gateway: 10.99.99.1
Domain name: vmlab.lan
Default lease time: 7200 (2Hours)
Maximum lease time: 14400 (4Hours)
Time format change: Yes
NTP:  Yes
NTP Server 1: pfsense.vmlab.lan
NTP Server 2: 10.99.99.1
Save and Apply





As you can see from this page there is a lot of information and configuration parameters you can give your devices and computers through the use of DHCP Server that is not covered in common Home-Routers that you can pick up at your local electronics or computer shop so you need to do some research if you want to use some of the other options as I am only covering what is necessary for the services we have configured in this guide.

Static DHCP Mapping on LAN
MAC Address: 00:0c:29:fa:58:63
IP Address: 10.99.99.15
Hostname: admin-pc
Description: admin pc static ip
Save and Apply





As you can see there is a lot of options that can be assigned on a per device \ IP basis when you create a static IP assignment, but the 2 main thing to keep in mind is that MAC Addresses are unique to every network device and the IP Address need to be outside of what you assigned for the DHCP Range and those IPs are already in use by the server and would make it a IP conflict. And that is all there is to configure DHCP and or a Static IP in pfSense.

6. Email Reports and Alerts
Now I am going to cover something most people neglect to setup in their home or small networks and that is email alerts and reports on events whit the add-on Mailreport as it allows you to setup periodic e-mail reports containing command output, and log file contents, this is a very useful tool to use and see what is going on whit your firewall and network so that you can be ahead and proactive against any issues that are on the rise and all this from the comfort of your email inbox.

Now since this is about sending emails the configuration for this package is very dependent on what email provider you have and I have gotten this to work whit the following providers: Google Gmail, Microsoft Hotmail (live\outlook), Yahoo Mail and Mail.com and it will work for any provider that allows you to connect to it using SMTP and IMAP applications like Thunderbird or Offic-Outlook.

Before we start configuring the notifications and mailreport features I strongly recommend that you have the following information ready:
IP Address of E-Mail server: IP address or FQDN of the SMTP E-Mail Server
From e-mail address: E-mail address that will appear in the from field. (Ex. hostname@domain.com)
Notification E-Mail address: E-mail address which will receive email notifications
Notification E-Mail auth username (optional): Username for SMTP authentication
Notification E-Mail auth password (optional): Password for SMTP authentication

When looking for this information you want the Outgoing SMTP and IMAP server settings from the provider, for your benefit I will add the provider settings I have used in the past below, We start the configuration whit setting the notifications details under System -> Advanced -> Notification

Google Gmail
smtp.gmail.com
Requires SSL: Yes
Requires TLS: Yes (if available)
Requires Authentication: Yes
Port for SSL: 465
Port for TLS/STARTTLS: 587
Account Name, User name, or Email address: Your full email address
Password: Your Gmail password



Microsoft Hotmail (live\outlook)
smtp-mail.outlook.com
Requires SSL: Yes
Requires TLS: Yes (if available)
Requires Authentication: Yes
Port for SSL: 587
Port for TLS/STARTTLS:
Account Name, User name, or Email address: Your full email address
Password: Your Hotmail, Live or Outlook password



Yahoo Mail
smtp.mail.yahoo.com
Requires SSL: Yes
Requires TLS: Yes (if available)
Requires Authentication: Yes
Port for SSL: 465
Port for TLS/STARTTLS: 587
Account Name, User name, or Email address: Your full email address
Password: Your Yahoo password



Apple iCloud Mail
smtp.mail.me.com
Requires SSL: Yes
Requires TLS: Yes (if available)
Requires Authentication: Yes
Port for SSL: 587
Port for TLS/STARTTLS:
Account Name, User name, or Email address: the name part of your iCloud address (so without @icloud.com behind it)
Password: Your iCloud password



MAILcom
smtp.mail.com
Requires SSL: Yes
Requires TLS: Yes (if available)
Requires Authentication: Yes
Port for SSL:
Port for TLS/STARTTLS: 587
Account Name, User name, or Email address: Your full email address
Password: Your Mailcom password



Now make sure you save the settings before you test the SMTP settings. You should see a green bar at the top saying successes if everything is working if not it will be red whit an indication of what went wrong, in most cases that would be a case sensitive username \ password or that the provider changed ports.

Now that we have enabled pfSense to email us we can continue to install the Mailreport package from System -> Packet Manager
When at the Package Manager -> Available Packages search for "mail" and Mailreprter should be the only hit, now click install



You should always be prompted to confirm any installation so click on the green confirm button



Now some text should fly over the screen and a progress bar should follow until the installation is done, this should only take a minute or so to complete and you should see a installation success message when done.



When the package is installed it would be accessible under Status -> Email Reports
From here you need to add a report schedule before you can add what types of logs it should send you, I usually have it set to once a week on every Sunday at 05:00 in the morning since there should not be any activity on the network at that time in my case

Report Settings
Description: Weekly Report

Schedule
Frequency: Weekly
Day of the Week: Sunday
Hour of Day: 05
Save



Now we need to go back and edit the Weekly Report and add what logs it should send us, this is done by clicking on the pen icon under Actions



When your back at the Weekly reports you should see 2 new fields for Include Commands and Include Logs, I usually include the logs of System, Gateway Events and Firewall to get a overview of what is going on there
Send Now will send you a copy of the current setup to review, then Save to store the configuration



That is it for this guide.

If you follow this guide and it is not working for you and it broke your system, I am not responsible or liability for that as you should not take anything you read on the internet at face value and you should test settings like this in a lab environment and not on your production servers

15
Edd Noman's Guide to pfSense 01 - How-To Install and Configure pfSense 2.3 inside a VM using VMware workstation

This is not an in-depth guide for Networking, pfSense, or VMware Workstation. It will provide you with a basic working setup covering the most important components you need to configure 
For you virtual environment and pfSense to run smoothly.

1. Preparations, what do you need to follow this guide

1. PC with local Internet connection
2. VMware Workstation - http://www.vmware.com/products/workstation/workstation-evaluation
3. pfSense ISO file - https://www.pfsense.org/download/ 
4. Windows or Linux installation ISO file
5. Good time, you should have about 2 - 3 hours free to do this if you are completely fresh in the area. Do you have network experience before you can do this in about 1 hour

1.a PC recommended specification:
CPU: i5 4-cores at 2.5 GHz and supports VD-T, VD-X
RAM: 8 GB ++
HDD: 120 GB

You can run on lower hardware but then it will be a slow and painful experience, also AMD equivalent is ok

1.b Internet connection: I recommend you have minimum connection of 10/5Mbps or you are going to wait forever on the files to download, if you are on a slow connection I recommend 
Downloading all the needed files before you start on this project.

2. Network preparation and layout

In this guide, your normal home router and network will act as the ISP and the WAN connection for the VM LAB network that we create in Workstation, so we need to create a layout and map 
Out how everything is going to be connected together otherwise it is easy to get lost and then end up whit a broken configuration, and you need to start over from scratch since you have 
No point of reference to start troubleshooting the issues.

My home network layout and configuration is pretty simple, and it is what I am going to use as an example in this guide:

Home Router: D-link DIR-855
WAN: DHCP IP X.X.X.X  <- This is given from your Service Provider, you have no control of this
LAN Network: 192.168.1.0
LAN Broadcast: 192.168.1.255
LAN Subnet: 255.255.255.0 = /24 in CIDR notation as used in pfSense
LAN IP: 192.168.1.1
LAN DHCP: 192.168.1.100 to 192.168.1.200

PC whit VMware workstation installed has the local IP of 192.168.1.199

Now I want to define the VM LAB network before I start so we know what parameters to use when asked to provide the different network settings in the pfSense installer and the 
Workstation network configurator that we need to setup before we can create our Virtual Machines

LAB Router: pfSense 2.3
WAN: DHCP IP 192.168.1.xx <- the 1.xx is going to be the next unused and available IP from your Home Router in my case that should be 192.168.1.197
LAN Network: 10.99.99.0
LAN Broadcast: 10.99.99.255
LAN Subnet: 255.255.255.0 = /24 in CIDR notation as used in pfSense
LAN IP: 10.99.99.1
LAN DHCP: 10.99.99.100 to 10.99.99.200

Now you are probably wondering why I chose to go whit a 10.99.99.0 network and IP range for my VM LAB setup and there are 2 good main reasons for that

1. Your VM LAB network cannot be the same as your Home Router network that will only make problems and issues for anyone that connect to the internet from your home

2. If something is to go bad an you need to troubleshoot a network issue it is easy to identify that anything whit a 10.99.99.xx IP is from your VM LAB network and anything whit a 
192.168.1.xx IP is from your normal Home Router network that is having or creating issues.

3. Configuring the virtual network infrastructure in VMware Workstation

Now that we have done all of our preparations and network planning we are ready to start configuring and installing the VM LAB network, we start by configuring the network configuration 
Settings by opening Virtual Network Manager from Edit -> Virtual Network Manager

Here you should see 3 different types of networks already defined for you, those should be Bridge, NAT and Host-Only, all this is fine what we want to do is to Add Network and create a 
Custom Host-Only types of network for our LAN interface, to do this click on the Add Network button choose a "VMnet #" I usually go for VMnet3 then click OK



Now back at the Virtual Network Editor you should see 4 for networks in the list the same 3 as before and our newly created VMnet3, you would need to select that in the list and edit 
Its network values according to our VM LAB requirements witch is:

VMnet Info = Host-only
Connect to a host virtual adapter to this network = NO (not checked)
Use local DHCP service = NO (not checked)
Subnet IP = 10.99.99.0 (this is the LAN Network of the VM LAB environment)
Subnet mask = 255.255.255.0

Save these changes by clicking Apply then OK to go back to the main window of VMware workstation.



[Some info about the different types of networks used in VMware workstation for quick reference]
- Bridged type means that the virtual machine shares the physical network port and connection whit the computer, our pfSense will use this for its WAN interface.
- NAT type means it is a translated network that is given to the virtual machine, essentially workstation will act as its own router
- Host-Only type means that only that specific virtual Machine can use that network, it is used to emulate an isolated network whit no internet access or access to other devices

4. Configuring the Virtual Machin Settings and Hardware

Now is the time to define your hardware to be used for pfSense VM, to do this navigate to File -> Create a New Virtual Machine
Here we will be following the on-screen wizard so select typical and click next



Select I will install Operating System later, the virtual machine will be created whit a blank hard disk and press next



Set the guest operating system option to Other -> FreeBSD 64-Bit then click next



Now give the virtual machine a proper identifying name and where one the system you would store the system files then click next

Name: pfSense 2.3 Router
Location: C:\VirtualMachines\pfsense2.3\



Select the amount of storage and hard drive space to be used by the pfSense VM, this would depend on what you are going to use the system for some add-on packages and features would 
Require lager amount of storage than others, for a base system and learning pfSense anything between 5 GB to 10 GB is fine I select 10 GB as that is a nice round nr that allows for growth 
And space to test out most of the features and add-on packages whit ease

I choose 10 GB and single file, then next



At this point you should be looking at a summary page of all the options you have chosen and workstation recommendation of hardware for a FreeBSD system, however we are setting up a 
firewall and router system that does not need many of the hardware options like a normal system would have benefits of using so we would need to edit the selected hardware to fit our 
Install of pfSense, you do this by clicking Customize Hardware button



Now you should see the Hardware selection list, here you should set the following hardware and settings

Memory = 2048mb
Processor = 4 \ 2 processors and 2 cores per processor
Hard Disk = 10 GB
DVD = ISO Image: File location of pfSense iso file, mine is "C:\Users\noman\Downloads\pfSense.iso"
Network Adapter 1 = Bridged (Automatic)
Network Adapter 2 = Custom (VMnet3) 
Display = Auto Detect

As you can see I removed the sound card and usb controller since this is a Virtual Machine you do not have access to put an usb drive to it nor dose a router need a soundcard so removing 
This will save some resources of your system, however if this was a physical install on a machine I do recommend usb ports.
I have also added a second Network adapter whit different types of VMnet networks







When you have set the hardware to your liking you click Close to go back to the summary picture and then press Finish to complete the creation of the virtual machine and return to the 
Main menu of the VMware Workstation.



5. Installation of pfSense software

From this point is going to be a normal installation process for pfSense, and you can follow the normal installation guide from https://doc.pfsense.org/index.php/Installing_pfSense
However when the installation process is done this do not cover the optimizations steps needed for running smoothly in a VM environment

Whit that said we are now ready to start up the new virulent machine, which is done by pressing the green play button and you will open a new window that represents a physical display 
Connected to the machine you just created and the first menu in pfSense

The first screen that we get is the pre-boot option where pfSense ask what type of environment we would like to use, you have a 10 second window to select it manually after that the 
Autoboot will boot the default recommended environment for us



Now you will see a lot of text fly over the screen until it stops and prompt you to select between Recovery mode or Installation mode, again you have 10 seconds to select otherwise it 
Will start the installation mode

[Recovery mode is used to recover the configuration xml files from a non-working system before you wipe the drive and reinstall pfSense on to it]



Now at this screen it will ask you to set your current screen resolution, keyboard layout and video fonts, I just accept the default settings



Next it will ask you if you want to do a quick and easy install or if you want to set some advances options like raid and other custom hardware option, you will once again get the 
option to do a recovery of the system if you missed the first selection, since this is a brand new install as a VM I select quick and easy option



It will now ask for confermation on your settings and warn you about this option is going to wipe and formant the disk and all you files will be lost, dont worry you will not lose anything as it uses a virtual file as its hard drive, however if it had been a physical device it would have wiped all the files from it



Now that the installation has started it will progress up to 50% and then I will ask you for what kernel you want to use, the option are Standard Kernel and Embedded Kernel, No VGA the 
Only difference here is that embedded kernel is to be used on devices where you have no graphical connection for a monitor, so I select Standard Kernel





Now after the selection of what kernel to install it will finish up the installation process and when it reaches 100% it will ask you to reboot or drop down to the console and shell menu 
In case you want to make some last minute custom changes to the system, I choose reboot



Now if everything went ok whit the installation process you should see the same preboot options for multi user or single user mode, when the timer has expired it should fly a lot of text 
Over the screen and you should end up whit a screen saying "Welcome to pfSense 2.3.3" and then give you 16 different options like this



As you can see I have gotten an IP address of 192.168.1.198/24 on my WAN interface but I am missing the famous 192.168.1.1 IP on the LAN interface, the reason for this is that pfSense 
Default settings uses NAT and whit that enabled you are not allowed to have the same network on both LAN and WAN interface so we need to manually tell it what network and IP to use on 
LAN interface so that we can connect to it and configure it from the WEBGUI, to set a network and IP manually enter option 2 Set Interface(s) IP Address

[At this stage I have seen some issues where it mixes the LAN and the WAN interfaces so that it assigns the LAN interface as WAN and gives you the 192.168.1.1 IP on LAN but no IP on 
WAN, if that is the case you need to use option 1 assign interfaces]

You will now get a series of questions about what interface to change, what the new IP Address and Subnet mask for this interface would be, it will also ask me about Gateway options and 
DHCP Server and if we would like to set the WEBGUI access to HTTP instead of HTTPS access

I select the following

Option 2 Set Interface(s) IP Address
2 for LAN
IP: 10.99.99.1
Subnet: /24  - 255.255.255.0
Gateway: blank - press enter to skip
DHCP Start: 10.99.99.100
DHCP Stop: 10.99.99.200
Set WEBGUI to HTTP: NO





When all the option is set and pfSense is finished whit processing the new values you are given a summary page of what has been done in this case changed the LAN IP and enabled WEBGUI 
Access on https://10.99.99.1 and ask you to press enter to continue back to the shell menu



Now at the shell menu you should verify that WAN interface have the same IP range and network as your Home Router and your main computer 192.168.1.XX and that LAN interface have the 
correct IP that we just set to 10.99.99.1, if all this match up then congratulations we are now don whit the installation process and are ready to connect to the pfSense WEBGUI where all 
The magic and configuration happens



6. Configuration of a LAN client

So far we have installed and configured pfSense inside a VM in VMware workstation, but we have no access to actually connect to it and use its feature in any meaningful way, so the next 
step needed is to create a second VM and install your favorite desktop OS, what you use here should be the OS you prefer for simplicity I use a Window 7 install whit Firefox installed 
On it so everyone can follow, I use Widows 7 as that is what I have access to and a license for when writing this guide

For my Windows 7 LAN client I use the following setup

Memory = 4096Mb
Processor = 4 \ 1 processors and 4 cores per processor
Hard Disk = 25 GB
DVD = ISO Image: File location of pfSense iso file, mine is "C:\Users\noman\Downloads\Win7.iso"
Network Adapter = Custom (VMnet3) 
Display = Auto Detect
 
You would need to adjust these settings accordingly to what your selected OS has as a minimum requirement, the only important part here is that you set the Network Adapter to Custom on 
VMnet3 so that it will be connected to the same network as our pfSense LAN interface



Now when you boot your LAN-Client you should verify network connectivity and check that you have been assigned the correct IP and network, since I am using Windows 7 as my client 
Computer I verify this by using CMD.exe and ipconfig and ping command

1. Open CMD.exe
2. Run command ipconfig /all
Here you would look for the following details

IPv4: 10.99.99.100
Subnet mask: 255.255.255.0
Gateway: 10.99.99.1
DHCP: 10.99.99.1
DNS: 10.99.99.1

3. Check for internet connectivity using ping command

Ping 8.8.8.8

Ping google.com

This will check that you have the basic internet connectivity and that you can access google from both IP and DNS URL



If all of this is working as it should and you can access the internet we are now ready to start configuring pfSense from its WEBGUI

4. Connecting to pfSense WEBGUI for the first time

Now it is time for us to open up our favorite web browser and navigate to the pfSense WEBGUI using the following web address http://HTTPS://10.99.99.1/ now you would probably expect a login 
Page for pfSense but instead are met by an error message saying this site is not secure or the connection is not private, do not worry this is to be expected as pfSense uses a self 
signed SSL cert and any of the new web browsers have been set to not accept self-signed SSL certs, so you can safely ignore this error \ waring and move on



In Firefox you do the following:

1. Click on Advanced button
2. In this new box click Add exceptions



Finally you now should see a webpage whit the pfSense logo on it and asking for a username and password to login when you enter the username and password pfSense will ask you to follow 
A setup wizard and it is recommended that you follow it the first times you configure it.

7. pfSense Setup Wizard and General Settings

Now is the time we are able to login to pfSense whit the default username and password to start the online setup process of the Wizard

Username: admin
Password; pfsense



Since it's the first time you log in to the pfSense WEBGIU, you will be prompted to follow a configuration wizard, if you feel that you have a clear understanding of firewall and router 
Setups, you can skip out of it by clicking on one of the pfSense logos that appear on the screen, I recommend that you complete the wizard as the first time user of pfSense



On the next screen you get information about "Gold Subscription" for only $ 99 USD This year, this is strongly recommended when using pfSense in a production network, as you get things 
like auto backup feature, access to the full version of the book , As well as monthly hangouts with pfSense people who show tips and tricks on what and how to configure a lot of the 
Advanced features, but for a test setup like this you can skip this



Now you are prompted to set the general information about your system and asked about Hostname, Domain and DNS Server. I use the following settings

Hostname: pfSense
Domain: vmlab.lan
Primary DNS: 192.168.1.1
Secondary DNS: 8.8.8.8
Override DNS: NO (Unchecked)



Next we are asked to select our location and time zone for the NTP service to sync our clocks, I use the generic time server of 0.pfsense.pool.ntp.org and select the city closest to me 
That is Europe/Oslo, you have to select what is the closest one to you, and you can also find local NTP servers at ntp.org, more on that in a later guide



Now we get setup for the WAN \ Internet access, here you usually enter the information received from your ISP but in our setup it is your normal Home Router network that serves as 
Internet access so that information you set here depends on the settings you have on the Home Router. My recommendation is to leave it to default settings and only set the DHCP Hostname 
Value

Type: DHCP
MTU: 1500
DHCP Hostname: pfSense.vmlab.lan
Do not Block RFC1918 Networks (not checked)
Block BOGON Networks (not checked)



Next page you get questions about the LAN settings, since we already set this in the shell console, just go ahead. No things needed to be changes



Set new administrator password, this is completely up to you but is highly recommended



Press reload to load all the settings we set and move on



Confirmation that the wizard is complete and that pfSense is ready for use, click on the logo to get to the main page also called the dashboard in pfSense



Now you want to see a fancy status page, this is the dashboard page where you can quickly get information about what is moving on your server, this is also the first page you see when 
You usually log in.



8. Driver installation

If this had been a normal physical machine, we would have finished this guide now, but since we run this in a virtual machine, we will install VMware Tools that is the "driver" package 
For all the machines you install in VMware, go to System -> Package Manager



Select Available Packages and search for "open" then open vm tools will appear, then click install



You will now be asked if you confirm the installation of this additional package



It now wants to start working, you want to see a status bar at the top and some text flies over the screen while it is working



Get back to the dashboard so you can always press the pfSense logo, congratulations, you now have a fully functional installation of pfSense running in the VMware Workstation on a 
Separate test network.

This concludes this part of the guide series, I have content for a lot more guides and tutorial for pfSense packages and configurations that I will use this as a base system to work out of so everyone have a common point of reference to the core system and settings when we get into a lot more advances features of pfSense

If you follow this guide and it is not working for you and it broke your system, I am not responsible or liability for that as you should not take anything you read on the internet at face value and you should test settings like this in a lab environment and not on your production servers.


Pages: [1] 2