View source for Filter (software)

{{for multi|Internet filtering software|Content-control software|video filtering software|Filter (video)|other uses|Email filtering}}
A '''filter''' is a [[computer program]] or [[subroutine]] to process a [[Stream (computing)|stream]], producing another stream. While a single filter can be used individually, they are frequently strung together to form a [[Pipeline (software)|pipeline]].

Some [[operating system]]s such as [[Unix]] are rich with filter programs. [[Windows 7]] and later are also rich with filters, as they include [[Windows PowerShell]]. In comparison, however, few filters are built into [[cmd.exe]] (the original [[command-line interface]] of Windows), most of which have significant enhancements relative to the similar filter commands that were available in [[MS-DOS]]. [[OS X]] includes filters from its underlying Unix base but also has [[Automator (software)|Automator]], which allows filters (known as "Actions") to be strung together to form a pipeline.

==Unix==
In [[Unix]] and [[Unix-like]] operating systems, a filter is a program that gets most of its data from its [[standard input]] (the main input stream) and writes its main results to its [[standard output]] (the main output stream). Auxiliary input may come from command line flags or configuration files, while auxiliary output may go to [[standard error stream|standard error]]. The command syntax for getting data from a device or file other than standard input is the input operator (<code>&lt;</code>). Similarly, to send data to a device or file other than standard output is the output operator (<code>&gt;</code>). To append data lines to an existing output file, one can use the append operator (<code>&gt;&gt;</code>). Filters may be strung together into a [[Pipeline (software)|pipeline]] with the pipe operator ("<code>|</code>"). This operator signifies that the main output of the command to the left is passed as main input to the command on the right.

The [[Unix philosophy]] encourages combining small, discrete tools to accomplish larger tasks. The classic filter in Unix is [[Ken Thompson]]'s {{Mono|[[grep]]}}, which [[Doug McIlroy]] cites as what "ingrained the tools outlook irrevocably" in the operating system, with later tools imitating it.<ref name="reader">{{cite tech report |first1=M. D. |last1=McIlroy |authorlink1=Doug McIlroy |year=1987 |url=http://www.cs.dartmouth.edu/~doug/reader.pdf |title=A Research Unix reader: annotated excerpts from the Programmer's Manual, 1971–1986 |series=CSTR |number=139 |institution=Bell Labs}}</ref> {{Mono|grep}} at its simplest prints any lines containing a character string to its output. The following is an example:
<syntaxhighlight lang="bash">
cut -d : -f 1 /etc/passwd | grep foo
</syntaxhighlight>
This finds all registered users that have "[[foo]]" as part of their username by using the [[Cut (Unix)|cut]] command to take the first field (username) of each line of the Unix system password file and passing them all as input to grep, which searches its input for lines containing the character string "foo" and prints them on its output.

Common Unix filter programs are: [[Cat (Unix)|cat]], [[cut (Unix)|cut]], [[grep]], [[head (Unix)|head]], [[Sort (Unix)|sort]], [[tail (Unix)|tail]], and [[uniq]].
Programs like [[awk]] and [[sed]] can be used to build quite complex filters because they are fully programmable. Unix filters can also be used by [[Data scientists]] to get a quick overview about a file based dataset.<ref>[http://blog.comsysto.com/2013/04/25/data-analysis-with-the-unix-shell/ Data Analysis with the Unix Shell] {{Webarchive|url=https://web.archive.org/web/20160122040316/http://blog.comsysto.com/2013/04/25/data-analysis-with-the-unix-shell/ |date=2016-01-22 }} - Bernd Zuther, comSysto GmbH, 2013</ref>

===List of Unix filter programs===
{{Div col|colwidth=15em}}
*[[AWK|awk]]
*[[cat (Unix)|cat]]
*[[comm]]
*[[compress]]
*[[cut (Unix)|cut]]
*[[Expand (Unix)|expand]]
*[[Fold (Unix)|fold]]
*[[grep]]
*[[head (Unix)|head]]
*[[Nl (Unix)|nl]]
*[[paste (Unix)|paste]]
*[[perl]]
*[[Pr (Unix)|pr]]
*[[sed]]
*[[Unix shell|sh]]
*[[Sort (Unix)|sort]]
*[[Split (Unix)|split]]
*[[Strings (Unix)|strings]]
*[[Tac (Unix)|tac]]
*[[tail (Unix)|tail]]
*[[Tee (command)|tee]]
*[[tr (Unix)|tr]]
*[[uniq]]
*[[Wc (Unix)|wc]]
*[[gzip|zcat]]
{{Div col end}}

==DOS==
Two standard filters from the early days of DOS-based computers are [[List of DOS commands#FIND|find]] and [[List of DOS commands#SORT|sort]].

Examples:

 find "keyword" < ''inputfilename'' > ''outputfilename''
 sort "keyword" < ''inputfilename'' > ''outputfilename''
 find /v "keyword" < ''inputfilename'' | sort > ''outputfilename''

Such filters may be used in [[batch files]] (*.bat, *.cmd etc.).

For use in the same [[command shell]] environment, there are many more filters available than those built into Windows.  Some of these are [[freeware]], some [[shareware]] and some are commercial programs.  A number of these mimic the function and features of the filters in Unix. Some filtering programs have a [[graphical user interface]] (GUI) to enable users to design a customized filter to suit their special [[data processing]] and/or [[data mining]] requirements.

==Windows==
[[Windows Command Prompt]] inherited MS-DOS commands, improved some and added a few. For example, [[Windows Server 2003]] features six command-line filters for modifying [[Active Directory]] that can be chained by piping: DSAdd, DSGet, DSMod, DSMove, DSRm and DSQuery.<ref>{{cite book|last1=Holme|first1=Dan|last2=Thomas|first2=Orin|title=Managing and maintaining a Microsoft Windows Server 2003 environment : exam 70-290|date=2004|publisher=[[Microsoft Press]]|location=[[Redmond, WA]]|isbn=9780735614376|pages=[https://archive.org/details/mcsaselfpacedtra00micr/page/3 3{{!}}17—3{{!}}26]|url=https://archive.org/details/mcsaselfpacedtra00micr/page/3}}</ref>

[[Windows PowerShell]] adds an entire host of filters known as "cmdlets" which can be chained together with a pipe, except a few simple ones, e.g. <code>Clear-Screen</code>. The following example gets a list of files in the <code>C:\Windows</code> folder, gets the size of each and sorts the size in ascending order. It shows how three filters (<code>Get-ChildItem</code>, <code>ForEach-Object</code> and <code>Sort-Object</code>) are chained with pipes.
<syntaxhighlight lang="powershell">
Get-ChildItem C:\Windows | ForEach-Object { $_.length } | Sort-Object -Ascending
</syntaxhighlight>

==References==
{{Reflist}}

== External links ==
* http://www.webopedia.com/TERM/f/filter.html

[[Category:Software design patterns]]
[[Category:Programming paradigms]]
[[Category:Operating system technology]]